{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>可用于产品的线性回归算法建流程</h3>\n",
    "<h4>数据准备阶段</h4>\n",
    "\n",
    "1.载入数据集\n",
    "\n",
    "2.数据预览\n",
    "\n",
    "3.获取特征数据和输出数据\n",
    "\n",
    "<h4>特征工程阶段</h4>\n",
    "\n",
    "4.缺失值处理，异常值处理，高级特征生成（若有）\n",
    "\n",
    "5.划分数据集为训练集和验证集\n",
    "\n",
    "6.训练集数据的标准化\n",
    "\n",
    "<h4>模型训练阶段</h4>\n",
    "\n",
    "7.使用不同的参数在训练集上训练出模型(使用训练集交叉验证是可选的)\n",
    "\n",
    "8.观察训练集的MSE和RMSE\n",
    "\n",
    "<h4>模型验证阶段</h4>\n",
    "\n",
    "9.验证集数据的标准化\n",
    "\n",
    "10.使用训练出的模型在验证集上做预测，并观察验证集的MSE和RMSE\n",
    "\n",
    "11.画图观察(可选)\n",
    "\n",
    "12.重复步骤7-11，选择验证集上MSE或RMSE最小的模型作为模型输出\n",
    "\n",
    "<h4>算法上线阶段</h4>\n",
    "\n",
    "13.保存模型，加载模型做预测,共3种方法\n",
    "    * 使用python pickle API 保存\n",
    "    * 使用python 加载pickle文件预测\n",
    "    * 使用sklearn joblib API 保存\n",
    "    * 使用sklearn joblib API 加载sklearn模型预测\n",
    "    * 保存为PMML文件\n",
    "    * 使用PMML Java API加载模型预测\n",
    "\n",
    "<h5>by Pinard Liu @ 20190131</h5>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>1. 载入数据集</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#载入必要的库，matplotlib用于画图，numpy用于数组操作，pandans用于数据处理\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read_csv里面的参数是csv在你电脑上的路径，此处csv文件放在notebook运行目录下面的CCPP目录里\n",
    "data = pd.read_csv('.\\ccpp.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>2. 数据预览</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AT</th>\n",
       "      <th>V</th>\n",
       "      <th>AP</th>\n",
       "      <th>RH</th>\n",
       "      <th>PE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.34</td>\n",
       "      <td>40.77</td>\n",
       "      <td>1010.84</td>\n",
       "      <td>90.01</td>\n",
       "      <td>480.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>23.64</td>\n",
       "      <td>58.49</td>\n",
       "      <td>1011.40</td>\n",
       "      <td>74.20</td>\n",
       "      <td>445.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>29.74</td>\n",
       "      <td>56.90</td>\n",
       "      <td>1007.15</td>\n",
       "      <td>41.91</td>\n",
       "      <td>438.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>19.07</td>\n",
       "      <td>49.69</td>\n",
       "      <td>1007.22</td>\n",
       "      <td>76.79</td>\n",
       "      <td>453.09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11.80</td>\n",
       "      <td>40.66</td>\n",
       "      <td>1017.13</td>\n",
       "      <td>97.20</td>\n",
       "      <td>464.43</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      AT      V       AP     RH      PE\n",
       "0   8.34  40.77  1010.84  90.01  480.48\n",
       "1  23.64  58.49  1011.40  74.20  445.75\n",
       "2  29.74  56.90  1007.15  41.91  438.76\n",
       "3  19.07  49.69  1007.22  76.79  453.09\n",
       "4  11.80  40.66  1017.13  97.20  464.43"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9568, 5)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>3. 获取特征数据和输出数据</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AT</th>\n",
       "      <th>V</th>\n",
       "      <th>AP</th>\n",
       "      <th>RH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8.34</td>\n",
       "      <td>40.77</td>\n",
       "      <td>1010.84</td>\n",
       "      <td>90.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>23.64</td>\n",
       "      <td>58.49</td>\n",
       "      <td>1011.40</td>\n",
       "      <td>74.20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>29.74</td>\n",
       "      <td>56.90</td>\n",
       "      <td>1007.15</td>\n",
       "      <td>41.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>19.07</td>\n",
       "      <td>49.69</td>\n",
       "      <td>1007.22</td>\n",
       "      <td>76.79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11.80</td>\n",
       "      <td>40.66</td>\n",
       "      <td>1017.13</td>\n",
       "      <td>97.20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      AT      V       AP     RH\n",
       "0   8.34  40.77  1010.84  90.01\n",
       "1  23.64  58.49  1011.40  74.20\n",
       "2  29.74  56.90  1007.15  41.91\n",
       "3  19.07  49.69  1007.22  76.79\n",
       "4  11.80  40.66  1017.13  97.20"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#获取特征数据\n",
    "X = data[['AT', 'V', 'AP', 'RH']]\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>480.48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>445.75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>438.76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>453.09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>464.43</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PE\n",
       "0  480.48\n",
       "1  445.75\n",
       "2  438.76\n",
       "3  453.09\n",
       "4  464.43"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#获取输出数据\n",
    "y = data[['PE']]\n",
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>4. 缺失值处理，异常值处理（若有）</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>5. 划分数据集为训练集和验证集</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(7176, 4)\n",
      "(7176, 1)\n",
      "(2392, 4)\n",
      "(2392, 1)\n"
     ]
    }
   ],
   "source": [
    "#将数据划分为训练集和验证集\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)\n",
    "print (X_train.shape)\n",
    "print (y_train.shape)\n",
    "print (X_test.shape)\n",
    "print (y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>6. 训练集数据的标准化</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AT      19.708303\n",
      "V       54.403151\n",
      "AP    1013.189242\n",
      "RH      73.220410\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "#对训练集数据特征进行归一化，保存均值向量和标准差向量\n",
    "norm_data_mean = X_train.mean()\n",
    "print (norm_data_mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AT     7.441593\n",
      "V     12.747985\n",
      "AP     5.898733\n",
      "RH    14.625541\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "norm_data_std = X_train.std()\n",
    "print (norm_data_std)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AT</th>\n",
       "      <th>V</th>\n",
       "      <th>AP</th>\n",
       "      <th>RH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>9103</th>\n",
       "      <td>0.441025</td>\n",
       "      <td>0.671278</td>\n",
       "      <td>1.086878</td>\n",
       "      <td>-0.558679</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6281</th>\n",
       "      <td>1.075342</td>\n",
       "      <td>1.564031</td>\n",
       "      <td>-2.551444</td>\n",
       "      <td>-0.093706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6201</th>\n",
       "      <td>1.380405</td>\n",
       "      <td>1.701317</td>\n",
       "      <td>-0.944208</td>\n",
       "      <td>0.162712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2646</th>\n",
       "      <td>-1.362478</td>\n",
       "      <td>-1.144037</td>\n",
       "      <td>-0.011739</td>\n",
       "      <td>1.089238</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3568</th>\n",
       "      <td>1.510763</td>\n",
       "      <td>1.463616</td>\n",
       "      <td>-0.733979</td>\n",
       "      <td>0.112112</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AT         V        AP        RH\n",
       "9103  0.441025  0.671278  1.086878 -0.558679\n",
       "6281  1.075342  1.564031 -2.551444 -0.093706\n",
       "6201  1.380405  1.701317 -0.944208  0.162712\n",
       "2646 -1.362478 -1.144037 -0.011739  1.089238\n",
       "3568  1.510763  1.463616 -0.733979  0.112112"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from scipy.stats import zscore\n",
    "X_train_norm = X_train.apply(zscore)\n",
    "X_train_norm.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>7. 使用不同的参数在训练集上训练出模型(使用训练集交叉验证是可选的)</h1>\n",
    "    * 不考虑正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#训练模型，不考虑正则化\n",
    "from sklearn.linear_model import LinearRegression\n",
    "linreg = LinearRegression()\n",
    "linreg.fit(X_train_norm, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[454.21842252]\n",
      "[[-14.68689837  -2.96103407   0.40905751  -2.31169185]]\n"
     ]
    }
   ],
   "source": [
    "print (linreg.intercept_)\n",
    "print (linreg.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>8. 观察训练集的MSE和RMSE</h1>\n",
    "    *  不考虑正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE: 20.99974553734626\n",
      "Train RMSE: 4.582547930720011\n"
     ]
    }
   ],
   "source": [
    "y_train_pred = linreg.predict(X_train_norm)\n",
    "from sklearn import metrics\n",
    "# 用scikit-learn计算MSE\n",
    "print (\"Train MSE:\",metrics.mean_squared_error(y_train, y_train_pred))\n",
    "# 用scikit-learn计算RMSE\n",
    "print (\"Train RMSE:\",np.sqrt(metrics.mean_squared_error(y_train, y_train_pred)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>9. 验证集数据的标准化</h1>\n",
    "    * 只需要做一次即可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AT</th>\n",
       "      <th>V</th>\n",
       "      <th>AP</th>\n",
       "      <th>RH</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5014</th>\n",
       "      <td>-0.256437</td>\n",
       "      <td>-2.871289</td>\n",
       "      <td>-0.759357</td>\n",
       "      <td>0.360984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6947</th>\n",
       "      <td>1.329245</td>\n",
       "      <td>-1.945653</td>\n",
       "      <td>-0.293155</td>\n",
       "      <td>-1.403737</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9230</th>\n",
       "      <td>-1.162157</td>\n",
       "      <td>-3.400000</td>\n",
       "      <td>1.437725</td>\n",
       "      <td>0.493629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4290</th>\n",
       "      <td>1.391059</td>\n",
       "      <td>-1.909569</td>\n",
       "      <td>0.752833</td>\n",
       "      <td>-1.342201</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6477</th>\n",
       "      <td>0.023073</td>\n",
       "      <td>-2.708126</td>\n",
       "      <td>-0.155837</td>\n",
       "      <td>1.283343</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AT         V        AP        RH\n",
       "5014 -0.256437 -2.871289 -0.759357  0.360984\n",
       "6947  1.329245 -1.945653 -0.293155 -1.403737\n",
       "9230 -1.162157 -3.400000  1.437725  0.493629\n",
       "4290  1.391059 -1.909569  0.752833 -1.342201\n",
       "6477  0.023073 -2.708126 -0.155837  1.283343"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#验证模型\n",
    "#测试集基于训练集的均值和方差进行标准化\n",
    "X_test_norm = X_test.loc[:,['AT','V','AP','RH']]\n",
    "X_test_norm['AT'] = (X_test['AT'] - 19.708303)/7.441593\n",
    "X_test_norm['V'] = (X_test['AT'] - 54.403151)/12.747985\n",
    "X_test_norm['AP'] = (X_test['AP'] - 1013.189242)/5.898733\n",
    "X_test_norm['RH'] = (X_test['RH'] - 73.220410)/14.625541\n",
    "X_test_norm.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>10. 使用训练出的模型在验证集上做预测，并观察验证集的MSE和RMSE</h1>\n",
    "    *  不考虑正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE: 86.35903350105353\n",
      "Train RMSE: 9.292956122841296\n"
     ]
    }
   ],
   "source": [
    "#模型测试集验证，确认最终表现\n",
    "y_test_pred = linreg.predict(X_test_norm)\n",
    "\n",
    "# 用scikit-learn计算MSE\n",
    "print (\"Train MSE:\",metrics.mean_squared_error(y_test, y_test_pred))\n",
    "# 用scikit-learn计算RMSE\n",
    "print (\"Train RMSE:\",np.sqrt(metrics.mean_squared_error(y_test, y_test_pred)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>11. 画图观察(可选)</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4lOX18PHvmckkmQSyKSqEzRVcUNC4olZUpO5UVLRarRvFtmqtxapdXKpvbWnV2v7crXWr4tYUlxZF3BUtGBat4opKoIJkAbJn5rx/zDPDZOaZySRkkkzmfK4rF5n7WXIzhOfMvZ1bVBVjjDEmlqevK2CMMaZ/sgBhjDHGlQUIY4wxrixAGGOMcWUBwhhjjCsLEMYYY1xZgDDGGOPKAoQxxhhXFiCMMca4yunrCmyJrbfeWkePHt3X1TDGmIyyePHib1R1SGfnZXSAGD16NIsWLerrahhjTEYRkS9SOc+6mIwxxriyAGGMMcaVBQhjjDGuLEAYY4xxZQHCGGOMq4yexWSMMX2psqqa2fNWsLquiWElfmZNGcPUCeV9Xa0eYwHCGGO6obKqmiufWk5TWwCA6romrnxqOcCACRLWxWSMMd0we96KSHAIa2oLMHveij6qUc+zAGGMMd2wuq6pS+WZKK0BQkRWishyEVkiIoucsjIReUFEPnb+LHXKRURuFZFPRGSZiOydzroZY8yWGFbi71J5JuqNFsQkVR2vqhXO6yuAF1V1Z+BF5zXA0cDOztcM4PZeqJsxxnTLrClj8Pu8Hcr8Pi+zpozpoxr1vL7oYjoRuN/5/n5galT5AxqyECgRkaF9UD9jjOnU1Anl/PakcZQ7LQavSGQMorKquo9r1zPSHSAUeF5EFovIDKdsW1VdA+D8uY1TXg58FXXtKqfMGGP6pakTyiMtiYAqsHk200AIEukOEBNVdW9C3Uc/EpFDk5wrLmUad5LIDBFZJCKL1q1b11P1NMaYbhnIs5nSGiBUdbXz51rgH8B+wNfhriPnz7XO6auAEVGXDwdWu9zzLlWtUNWKIUM6TWdujDFpNZBnM6UtQIhIoYgMDn8PHAW8B8wFznZOOxv4p/P9XOAsZzbTAUB9uCvKGGP6q4E8mymdLYhtgddFZCnwDvCsqv4buBGYLCIfA5Od1wDPAZ8BnwB3Az9MY92MMaZHDOTZTGlLtaGqnwF7uZSvB45wKVfgR+mqjzHGpEM4rcZAzMlkuZiMMWYLTZ1Q3mlAyMTEfhYgjDEmjSqrqrn26fepbWyLlGVKYj8LEMYY00XRrYFivw8RqGtsi2sZxGZ8jRaeCmsBwhhjBojYh35dU8eWwaVzlvCTOUvwikQWzyXSnamwvdlVZQHCGJO1uvOwdVsYFy0cEjoLDuFzJ964IOHPja3f6K38vPlpTeRnpLurygKEMSZrxHYNNbS20xbomCIDkj9se3oBXKKf67Yh0ecff0DDf1+h5NCzEAkln0hnV5XtB2GMyQrhB251XRNKqGsoHBzCUkmRkY4FcG4/N7qlEmzeRM38O1lz38VsWPg4TR8v7HBuulZtWwvCGJMVOusaCquua+rQ7RPbzTNp7BCeXFyd0r26IvYhv7quCdUgDe8toPblvxFsrIscq3nxbvK33xuPLw9I36ptCxDGmAEt/ICv7sKn7HC3z6IvajoEg+q6Jp5cXM20fcp5aOGXKd9PcMk8GiPf52HijQsigchXu5Ivn/kLLas/jDs3sGEtjSteZ9AeRyCQtlXbFiCMMQOC24AzkHCaaWea2gI88vZXcYPNTW0BXvpwHWceMDJpkBABVSgv8acUnJraglTXNRFo2sCyef/HpiX/xi2s5BRvS+kRM/DvtB8CnHHASJvFZIwxbpItRMv3eZIGB59HGJSf0+HaaIlmIq2ua+L6qeMAXIMIQH6Ol9+eNI6pE8qZeOOCToOEBgNsWvY8da88QLB5Y9xxycmlaP+TKdp/Gh5fHuW9sBrbBqmNMRmrsqqaWY8vdX3AN7UFEj74IdTtM/uUvaj69VGRXeFiedx2qWFzn//1U8fxx1P3ct3MJnrgedaUMfgS3Qxoqf6Q/z14GTXz/s81OPh3PoCh591GycHfxePLQ4A3rjg87YvsrAVhjMlY18x9n7Zg5+sN3ERfNWvKGNeuKLdbez1CQ0s7o694ttPFcB0Gnl3ig7a3sv7522hYPt/1+pzSYZQd+QP8O+zToby3UolbgDDG9DuJFrDFlkevYu6O8PqB6Iys1XVNSQeVA0GN/NzOFsOFH+Sz562Im1ILgNdHe/3XccXiy6Ns4mkMrpiKen0djvk80mupxC1AGGP6TCoDy9V1Tcx6YilXPrWMprZg5NquzEpKJNH6ge61STqKnl2U6OeICGVH/oA1910MGvq7FYw9hF1P/CG/nH4IAL/4x3IaWgORe07fb0Sv5W8STWE5eH9VUVGhixYt6utqGGNipJLCwi2Rnd/nJd/nSTp20JNKC3wU5OZ02mrojlyvUJiXQ11jG55OuqJqXryblpVV/OL62fz6B6dGyhO9R+HB7+4SkcWqWtHpeRYgjDE9ye2hFp6OGZ75A6Q0s2cg0EAbGxbNJdhQR+nh57meU5wT4Fcn7MnJ+43uUJ7oPSov8fPGFYd3u06pBoi0dzGJiBdYBFSr6nEicjjwByAXWAycp6rtEkos8ifgGKAR+L6qvpvu+hljepbbimUFHl74JRWjyiKffNOVHqI/afq8ipr5d9JeswoQCnb7Fnnb7RRprcROVY1teSUKoL313vXGGMQlwAdAkYh4gPuBI1T1IxG5DjgbuBc4GtjZ+dofuN350/Qzmbgzluk9yfr1L3tsKRBKSpfsAZjp2jespfbFe2j86M2oUqXmhdvZ7szZ+HN9/PakUGtq9rwVXDpniWvywETdXr01iymt6yBEZDhwLHCPU7QV0KKqHzmvXwCmOd+fCDygIQuBEhEZms76ma6LTXgWXpBUWVXd11Uz/USyh1dANfL7MmvKGPw+b4fjsa8zjba3Uv/mHFbffWFMcAhpX7+K9ppQ6o5rn36/0+SBSvzsWL/P22uzmNK9UO4W4HIgPPXgG8AnIuG+r5OBEc735cBXUdeucspMP+LWfZBKBkyTPWZNGeO6cCwsOj31b08aR2nB5mmcze09mwCvNzV++h9W3/sj6l57EG1viTkqDNrzKIbNuAvfVqFHXm1jW0opQMJdUeL8uaUD1F2Rti4mETkOWKuqi0XkMABVVRE5DbhZRPKA54H28CUut4lrXYnIDGAGwMiRI9NRdZNEou6DbOhPNslFdz0W5HojUzPdRHctNUdNXc2UOTPR+ZXaatdQ++JdNH36H9dzc4fuTNmRM8kb1r1P/Vs6IL0l0jkGMRE4QUSOAfIJjUE8pKpnAocAiMhRwC7O+avY3JoAGA6sjr2pqt4F3AWhWUzpq75xk6jfuLf6RE3/FDtzqaE1gM8r7ovDoq5JNQV3f1Li9/HGFYfz8znvcOetN1P/9pMQiJ+W6/EXUfKtsxm052RCw69d15vdSW7S1sWkqleq6nBVHQ2cBixQ1TNFZBsApwXxc+AO55K5wFkScgBQr6pr0lU/0z2J+o378pfY9D23B32y4AAw6/GlPTpI7eulzHJ1TW3sfdn93HfpNOrffDQ+OIiHQROOZdgFdzJ4ryldCg4+j1Ba4OuT7iQ3fbGSepbT/eQBblfVBU75c4SmuH5CaJrrOX1QN9OJ6JQENoup/+mJGWbduUd3uhi7m0MprLzE36GOsRld02m9p5jalvj655XvStnkmeRuu2OX79kb2Vm7yhbKGTNA9MSq2+7eY8J1z/fawxk298tHB7PefpI1ff4uax/7NQCewhJKDzuXwt0nRfaK7oreHmfoNwvljDG9I9kMs1QDRHfv0ZufM8Ndmm7BrKepKmgQ8cRPv/VvvzcFYw/BO6gslIY7r7BbP6M/d9FagDBmgOiJVbfdmaVWWVW9xVlVu2LvkcVMnVDOhOueT2twaF23kpr5d+Lffm+KDzjF9ZytT7g8aYuhvMRPY2t7wtZVf+xWimYBwpg06O3V5pVV1T2y6rars9TCn+J70xuf1jD6imfTdv9gSwN1r/+djYufBg3SuuYjCnebRE7R1nHnJgoOZ0blnUpXwr3eYAHCmB4W+0AIrzYH0vZAmD1vhWtw6OqG9pPGDuHhhV92uJcQ+jtMvHEBk8YO4aUP10UCX2Nre8ZNU01ENUjD+y9R+/J9BBvqNpe3tVD70r0MOfHnnd7DLSlhJk/ssABhTA/ribGArkqW/yiVn+m2r3P0PSAUJB5a+GWkfCDlUWr9+jNqXridluoPXI8HWxrQ9lYkJ9f1uEDSB3/0pkSZxAKEMT2sL1abJ+oaSrTXcrTeGOztrwJNG6l77SE2LflXZMOeaN6ibSg74nz8Ox+YsDupOzOQMiXhpQUIY3pYX6w2d9tTOdnsmOgHVGeb2QxEqkE2LX2eulcfINi0If4Er4/i/U+m6IBpeHz5ka6jJxdXp/weJ9IXXZDdZQHCmB7W1Yd1T+hKP3fsAyrbgkPL6hXUzL+D1jUfux7377QfpUfMwFeyHRBa3Tz7lL2YOqGcilFlW/zJvy+6ILvLAoQxPayvBiVT7efOxPxHPWXT+y+x/pmbcJvvlVMylNIjZ1Cw4754nVZV7DTUnhhLyKSElxYgjEmD3hyUjO3Pjp1pFBuc+uODqLf4d6jA4x/coVtJcvIoPmg6RftOJTcvj9kn75XWf7tMSnjZS+mtjDHp4LaB00MLv0y6oVN/fBD1Fq9/MCWHnhV5XTBmIsMuuJ3iA09FcnIpzM1Je2DPpISXFiCMyWCpdBfFbujk9oAaaAKN9STKMzdoz8kU7PYttpl+PUOmXklO0TaRY/W9sCI8vFFSX20C1BXWxWRMBku1uyj6vPCD6LLHlg64AWoNtLPx3Weoe/1htpryYwp3+1bcOeLxMuT4Wa7X91brKlPWRViAMCaDJerPdjsvWvjhNJDWPzR/sYya+XfQ9k1oMV/tS/fi32k/PLmpPfTT2c2TKeseYlkXkzEZLJXuokQPvnBXh7cb6an7k/YN37Dun7/j60evigQHgMCmGurfnJPwuok7lvVKN4/bOFHsuFB/ZS0IYzJYZ91FXpHIgy/RbKdM7WbS9jY2LKqk/s1H0baWuOOSV0hO8TaUl/gZvZWfhZ/VElDFK8Lp+4/okC8pnTJp3UMsCxDGZIhE3RRTJ5Rz6ZwlrtcEVZk6oZwz7n6LNz6tiZTH5lXKNE2fLabmxbtor3H/FF447khGf/t83vv99F6uWbxMWvcQK+0BQkS8wCKgWlWPE5EjgNmEurc2Ad9X1U+cPaofAPYB1gPTVXVluutnTCboLD1DorEIhbSmxu5t7fVfU/Pi3TR9vND1eO62O1I2+ULyysfS0Mt1SyST1j3E6o0xiEuA6BSJtwNnqOp44O/AL53y84BaVd0JuBn4XS/UzZiMkKib4rLHlrL9Fc/S2NqOz5PZYwnJBNtaqHvjEVbfc6FrcPDkD6Zsyo/Y7qybyCsfC4AI/aKfP5PWPcRKawtCRIYDxwI3AD91ihUocr4vBlY7358IXON8/wTwFxERzeRNs43pIYm6I8LjB7WNbfi8AzdA1M6/k03Lnnc5IgwaP4WSQ8/C6y/qcCSo9IskeLYfRGK3AJcDg6PKzgeeE5EmYANwgFNeDnwFoKrtIlIPbAV8k+Y6GtNvhccdUvmU1BYYuJ+livafxqb3F0CgPVKWO3QMZUddSN52OyW8rr8MBmfKuodYaetiEpHjgLWqujjm0KXAMao6HLgPuCl8ictt4n7jRWSGiCwSkUXr1q3r0Tob059ET4/Mdr6ycor2/Q4AnoJitjrmJ2z3vdlJg0NYJgwG91fpbEFMBE4QkWOAfKBIRJ4Fxqrq2845c4B/O9+vAkYAq0Qkh1D3U03MPVHVu4C7ACoqKgbuRyYz4LnNSgKydp8GVaV1zUfkDXPvmy8+cDqIULzfSXjyB0XKS/w+GlrbE7agMmEwuL9KW4BQ1SuBKwFE5DDgZ8BU4H8isouqfgRMZvMA9lzgbOAt4GRggY0/mIHKbVbSrMeXgmzuKsqm4ND6zZfUzr+D5i+Wse0Zs8kfvmvcOZ7cfEqjEu2VFvio+vVRQOj9vGbu+9TF5FLKlMHg/qpX10E4YwsXAE+KSBCoBc51Dt8LPCginxBqOZzWm3UzpjeEWw1u3UZtwcwMCB4JDQh3R7Clkbo3/s7GxU9DMBQsa+ffwXZn3YR4Eq8Q93mEq4/fPfI63MefqSkt+ivJ5A/pFRUVumjRor6uhjEdVFZVc+3T71PbGPo0W+L3cc0JoYfZQMp9tCVUlYb/vkzdS38l0FAbd7zsqB8yeMIxrtcKcPP08fbg3wIislhVKzo7z1ZSG9ODKquqmfXE0g794XVNbfz0sSWouu1jln1a135GzQt30LLqv67H80eNJ3+EexoMn1fSvqGP2cwChDFdlKgbo7KqOmFOpAztPepRgeZN1L/2EBurngMNxh33Dh5C6RHnU7DLQYhLAsHSAh9XH7+7BYdeZAHCmC5IlPJi0Rc1PLm4OqsGllOlGqRh+XxqX7mfYGN9/AneHIr3m0bRgafg8eXHHY7dF9r0HgsQxqQg2eByU1uAR97+yoKDi5Y1H1Pzwu20rvnI9bh/hwpKj5yBr3RY3LHoWUqmb1iAMCaJ2AHnRCw4xFNV1v/7VtrWfh53LKdkO0qPmEHBTvu5Xuv3eTvMUjJ9wzYMMiaBcHdSZ8HBuBMRyo78QceynFyKDz6DYefdFgkOQu9t3mO6xloQJqN1Z957qte4ZVA1XZM/Yg8KdvsWjf99Bf8uB1J2+PnkFG+LEJrRZeML/ZsFCJOxOtsjwc0vK5fz8MIvI9NNk11jOXxSE2iopWX1Cgp2PsD1eOlh5zJo98PZae+DbQFbhrEAYTJWV7dyrKyq7hAcOrumpMBn3UtJaDDAxnefoe61h9FAG8POvx1fyXZx5+UM3opRI4bzxhWH90EtzZawAGEyVle3ckyWNjt8TbLZSmaz5i+XU/PCHbR980WkrPbFu9lm2q/izvV5JJIPyVJhZBYLECZjdXUrx2RdRsNK/HFdVj3B55GMzbHkpn3jN9S+dB+NH7wSd6zpk7dpWrkE/+jxkbJwmpHwQsKudgmavmWzmEzG6upWjokChzj3SsegdDg4ZPpebxpoo/7tJ1h990zX4CC5BZQefgH5I/aIlJWX+Fly9VEddlRL1CVo+icLECZjTZ1Qzm9PGpfy9Ei3gCLAGQeMZOqE8rQOSmdyG6Lp8ypW//Ui6l7+G9rWHHe8cI8jKL/gTor2PRHxhjol3AJ1V7sETd+zLiaT0bqylWNnewMn6rJy4/MKOR6hqS0+p9BA0V6/ltoF99D40Zuux33b7EDZ5JnkD9+N0gIfBbk5SccWutolaPqeBQgzYCUaEE0UUCaNHeI6y8mVMmCDg7a3Uv/OU2x463G0vSXuuCd/ECWHnsWgvaYgHi8+r6SURG/WlDFxYzy2oU//ZgHCDBjRAaE4ZhvKzgZEK6uqeXJxdUrBwSsDa+A5lgba2PjuMy7BQRi011GUHHoW3oJioGsZVjtrwZn+J+mGQSLy02QXq+pNPV6jLrANg0z0tNTw6txkykv8rvPxx1/7fNx2lW78Pm9WrK7e9N6LrH/25sjr3KG7UDZ5JnlDdwHgzANGcv1U9z0bTP+X6oZBnQ1SD3a+KoALgXLnayawW4oV8YpIlYg847x+TUSWOF+rRaTSKRcRuVVEPhGRZSKydyr3N9krPG0y3K+dymd6twHRyqrqlIKDV4Rp+2THp93C3SeRV74rHn8RZd++mO2+94dIcCjx+6gYVdbHNTS9IWkXk6peCyAizwN7q+pG5/U1wOMp/oxLgA+AIueeh4QPiMiTwD+dl0cDOztf+wO3O38a46o701KjB0S7uiguoMqTi6u79PP6K1Wl6aO3UA1SOPbguOMiHrY67jI8+YPw5g/q0Dqra2qz9QtZItVpriOB1qjXrcDozi4SkeHAscA9LscGA4cDlU7RicADGrIQKBGRoSnWz2Shrk6PFEID0RDf+kjVQOhealv/FWsf+zXrKv8fNS/cTrB5k+t5vpLtQsFB4ltntn4hO6Q6SP0g8I6I/IPQ78p3gAdSuO4W4HJC3VSxvgO8qKobnNflwFdRx1c5ZWuiLxKRGcAMgJEjR6ZYfTMQdTYt1QNEzzNS4MnF1VSMKsvKTK3Blkbq33yUDYv+CcHQ3z3YWE/d6w/HpeUO83mlw/7a0Wz9wsCXUgtCVW8AzgFqgTrgHFX9f8muEZHjgLWqujjBKacDj0Rf4vajXepyl6pWqGrFkCFDUqm+GYAqq6qpaYifghn+JfKK4DYJNfzJN5sebqpKw39fYfU9M9nwzlOR4BC2aek8Ag11kddekcjCw8LcxJ8hbf3CwNeVaa4FwAZVvU9EhojI9qoav1XUZhOBE0TkGCAfKBKRh1T1TBHZCtiPUCsibBUwIur1cGB1F+pnskRlVTWzHl/qOtU0N8dDS3sw6Q5v2ZSIr3XdSmpeuIOWr95zPZ4/ak9Kj5yJt7AkUhZU5fMbjwVg+yueTXhvW78w8KUUIETkakIzmcYA9wE+4CFCQcCVql4JXOlcfxjwM1U90zl8CvCMqkav258L/FhEHiU0OF2vqh26l0x2S2VQuaV9YC5e66pg8ybqXv87G999BjT+PckrHsKoYy+kefi+iHRsvEe3DBJ145UW+GyAOgukOkj9HeAEoAFAVVfjPq6QqtPo2L0E8BzwGfAJcDfwwy24vxlgujuonG1Ug2xaPp/qu2eycfHc+ODgzaHowFMZ9YO7OOWUkymI6UKKXdmcKCGi7RedHVLtYmpVVRURBRCRwq78EFV9GXg56vVhLuco8KOu3Ndkj2wcVO6qtvVfsf65P9Gy+kPX4/k77EPZETPwlZXTArz04Tp+e9K4pCubbfVzdks1QDwmIncSmnp6AXAuLlNXjekpsXmUrOXQOcnJo3XtZ3Hl3uJtKTtiBv6d9uvQnbS6rimlZIddSYhoBpaUAoSq/kFEJgMbCI1D/FpVX0hrzUxW6SyPUippNBLxeWCA5tXrIKd4G4oOOIX61x8GQHJyKdr/ZIr2n4bHlxd3vs1CMp1JdZD6d6r6c+AFlzJjtkjsTmNuaS8Uuh0ksmncunj/aTS89yK+IaMpPfx81z2iwbKomtSkOkg92aXs6J6siMleqY4vhINEVw2kvKuBhjq+ee5PNH2x1PW45OSy3Vk3sc1Jv+wQHEr8PkoLfCltrGRMWNIWhIhcSGg20Y4isizq0GDAfRcRY7oo1UVrXpGk6xsGMg0G2Fj1HPWvPUSwpYHW1SvIP+fWyA5usPn98fqLOlybKIOtMZ3prIvp78C/gN8CV0SVb1TVmrTVymSVVAahsyXNtpvmr96j5oU7aFu3MlLWtv5LNi5+mqL9Nq81DajGvU/WlWS2RNIuJlWtV9WVwJ+AGlX9QlW/ANpExDKtmh4xa8oYfJ7EnUcCTNunnPIsG1Rt37ieb57+A1///YoOwSGs4cPXid7PpcTv69Ie3cZ0JtVprrcD0fszNLiUGdMti76oSbpDmwIPLfyS0gJf71WqD2mgnQ2L5lL/5iNoa3zLypNXQPFBpzN4n+M7TFsVsSmppmelGiBEoz6qqGpQRGy7UrPFKquqeXjhlymdW9vY+aY+ma5p5RJq599J2/qvXI8PGX8keRPPwjsofsOeuix4f0zvSvUh/5mIXEyo1QChgev4FTnGdNHseSsG1Cyj7mrfsJbaBX+lccXrrsd9Q0ZTNnkm/hF7JByzKcmSFpbpPakGiJnArcAvCbX4X8TZk8EYN7EroSeNHcJLH66LS9cwUNJul5f4I3+3xtb2LrV2Nr23gJrn/w9ti09f7skrpOTQ7zFo/NGIxxt572Y9sTRun4ZNze1UVlVbF5PpManuB7FWVU9T1W1UdVtV/a6qrk135Uxmik6sp4RWQj+08MsOr698ajmVVdUDYjVv7NjIsXsOjUtwl4yvdJhrcBi051EMu+BOBu99HOIJ3W/S2CFMnVDuuk9DW1BtlzfTozpbB3G5qv5eRP6M++Y9F6etZiYjxLYUZk0ZwzVz3+90SmpTW4Br5r7P7sMGZ3SeJY9AfWNbpMUQDoYTdyxj5fomVjtBMZm88rEUjptMw/JQooLc7XambPJM8obFT0996cN1ANS7rDYH2+XN9KzOupg+cP5clO6KmMwQHRBKCnxsam6PzECqrmtKuJGPm7qmNt74NLOX0yT6q77xaQ2lBT5unj6eq55aRmNbENUgIu6N9tJvnU3Lqvco2m8ag/acHGkxxAoHgETjEAOhRWb6j6QBQlWfdv68v3eqY/oLt5YBwGWPLyXgPBXd+tlTDQ7ZoLaxjZ8+toRAUGn6eCG1r/wtlAJjqxFx53oLSxh2/h0JA0NYOADMmjKmQ/4qsEVxpud11sX0NElS2ajqCT1eI9PnYpPnhccMAsFgJDiY1LSsr6Zm/l00fx7amr1m/l1sc+p1cbu4AR2Cg0fiWyfRAcD2aTC9obMupj84f54EbEdom1GA04GVqfwAEfES6qKqVtXjJPQ/43pC244GgNtV9Van/E/AMUAj8H1VfbcLfxfTQ9yS52VrmotkfF6Jm0kUFmxtpv6tR9nwTiUE2yPlzSuraProLQrGHER5khQjqnDL9PGdbuZjAcGkU2ddTK8AiMhvVPXQqENPi8irKf6MSwiNZYQziH0fGAGMdRbcbeOUHw3s7HztT2jNhaXz6AM20Nm5wlwvrS55xFWVxg9fp/alewls/Cb+QvHQVrMKgDeuOJyJNy5IOJZgAcD0tVTTfQ8RkR3CL0Rke2BIZxeJyHDgWDruPnchcJ1qaLPcqOmyJwIPaMhCQrvXDU2xfqYH2UBn5xpaA3HjLa3rvuDrR3/BN3N/5xoc8kbswdBzbqX4wFPx+0L/9RLt+WxjCaY/SHWh3KXAyyISXj09GvhBCtfdAlxOKD142I7AdBH5DrAOuFhVPwbKgej8AqucsjUp1tF0gdsgdPjTqtsAqEks2NJI3Rt/Z+PipyEY/555B5VROuk8CnY9NDL28NuT9gRsLMEUKzufAAAgAElEQVT0b6luOfpvEdkZGOsUfaiq8St7oojIccBaVV0sIodFHcoDmlW1QkROAv4KHIL7XjBxHbwiMgNnFffIkSNTqb6JkWgQGjr2a1/22NKs3X8hFapKw/svUffyfQQaauNP8ORQtO+JFB84HU9eQcL7WFeS6a9S6mISkQJgFvBjVV0KjHQCQDITgRNEZCXwKHC4iDxEqGXwpHPOP4A9ne9XERqbCBsOrI69qarepaoVqloxZEinvVzGxbVPxy9ka2oLdFiFO3VCOcFOgkN3dncbSL55ejbrn73JNTjkj57AsHP/TOlh57gGh/BKcmP6s1THIO4DWoEDnderCM1ESkhVr1TV4ao6GjgNWKCqZwKVQHh7q28BHznfzwXOkpADgHpVte6lHlZZVZ0wT1Ds4HRnYxHd3QJ0oCjY5aC4Mm/REIZ85yq2OfU61/UOYbEB2Zj+KNUAsaOq/h5oA1DVJrr/bLgRmCYiywntVHe+U/4coQyxnwB3E8oYa3pYsodSbECYNLbzFlo2d0AVjJlI/iinAez1UXzQaQw7/3YKdjnIdZ1DLJstZvq7VAepW0XEj/M8EJEdgaRjENFU9WXgZef7OkIzm2LPUeBHqd5zIPpl5XIeefur0L7CIpy+/wiunzquR39GsodSQ8vmbKCVVdU88rb7ngTZJtC8CW/+ICC0a1tDSyi9iIhQeuRMNrz2IMWHnYuvtGuT7my2mOnvUg0QVwP/BkaIyMOExhe+n65KZaNfVi7noaiNcwKqkdcVo8q6NMulsqqaa+a+T52T0K20wMfVx+/O1AnlSfd/rmtq48qnlrPoixqeXFydVQPUfp+XafuUM+edryLTVwON9dS9cj+NHy9k2Pm3kz+oBJFQOhGvCAFVtt9pDLN+9Diz561ImnRQ6NjasqmsJhOIdjYQGWorDye0uvkAQr/rC1XVZRVQ76qoqNBFiwZGHsEdr3zO9YHsEcjLid+IPnav4fC01UQPKZ9XmL7vCJ5ZuiYSOExIeVTQrayq5urKZax662nqXnuQYPMmAMr2Ppqyb/+4w8rp6H+H2JlhiX6OTWU1/YGILFbVis7O67QFoaoqIpWqug/wbI/UzsRJ9Gk9qPFpLsIDnOEHTCoPp7aA8vDCL7N6zCA2NYZboB3SuBLP3KuoWbKkw7U1Vf8md4/J5A3dJVIW/e/Q2dTg8hI/b1xxeFy5Mf1ZqoPUC0Vk37TWJMt5UxjUjBY9luCWO8lNNgeHMw8YyeyT96K8xI8Q6nbLy/Fw6ZwlTLxxAfe9UMXZZ5/NwQcfzJKY4ACQU1ruuggu+t9h6oRy/njqXrYy2gwYqY5BTAJmOmsaGnC6VFV1z6RXmZSdvv+IDmMQYX6fh6a2+Jw/0QOcNhumcy99uI7rp8Z3B2mgnQ9eeITzr36YYGv8+zho0CCmnf8TXs7ZB7zxez7HDjTbymgzkKQaII5Oay1MZLZS7CymilFlSfP+V1ZV43EGTE1ibi2u5i+WUTP/Dtq+iQ/MAN/97nf5/e9/z6kPrgCXICzg2jKwldFmoOhsP4h8YCawE7AcuFdV25NdY7rv+qnjEk5rdftEGv4kbMEh1Fca387aLPqT/pdffkXNS/fS+OFrrueOGzeOv/zlLxx6aCiB8eq6+C4nCHXZWSAwA1lnLYj7CS2Oe41QK2I3Qum7TS9K9Ik01bGHbJAsOMDmT/qPv/0Zax641DU9hievgFGTz2HDrkdx5ZvtzBocWhNS7Pe5zvwq8cd3ORkzkHQWIHZT1XEAInIv8E76q5TdkmVZjT1u7YbUlPh9kRbXr5/5iMH7foe6l//a4ZzCcUey1aTvE/SXAB0TGCaaP1DX1MbEGxfYGIMZsDoLEJGPTarankr6ANN9nWVZTWU6q+nI7/NyzQm7A5tbXEUVx7Np2fO016wid9sdKZt8IXnlY+OuDU9jrUuQuwri/42MGUiSLpQTkQChWUsQGpPzE1owF57FVJTo2t4wkBbKAQl3FwvPoU903MQLtrUQ3LCO3K2HE1QiK5/Dmr9YRlttNYP2PKrDXtCxBJKuPg+zdQ4mk6S6UC7pOghV9apqkfM1WFVzor7v0+AwECWarhout+ms7jyEFsGBs+Xnx2+z5t4f8vWT1xJoC336jx3Izx+1J4PHH500OACRbr7YtQ2x7N/GDESpTnM1W6CzcYWwZJ9Ut7/iWZvOmkAQyPcITetWUfPi3TR9trlVueE//6D4wFO7dd/wdOLotQ2J/n0s8Z4ZiDrNxdSfZUIXk9u4gc8rFObmUN/UFjdtddbjS+P2OjbJBVubqV/4GBveeQoCHWdhS04ewy64nZyibSJl5UkCcWGul8bWQMJA7vbv6Zayw5j+rMdyMZkt4zYVtS2gkWmT1XVNzHp8aeRYewYH7N6mqjSueIPaBfcS2Lgu/gTxMGivo/Dkbt7RzSuSdDynpCCX969LPJZgK6VNNrEAkWap9E23BZWrnlqGIlh8SE3bN19RM/8Omr9Y6no8b/julE2eSe4223coP33/0C5vnY33JGMrpU22sACRZqnMgAFodMm3ZOIFWxqpf/NRNiz6p2vyPO+gMkonnUvBrt+K29XtzANGRlaqJ/p3sbEEYzZLNZtrt4mIV0SqROQZ5/XfRORzEVnifI13ykVEbhWRT0RkmYjsne669YZUZsCY5DwS6k7a9P5LrL5nZmisITY4eLwU7XcSw86/g8LdDnPd8vOlD9dRWVUNuP+7WNZVYzrqjRbEJcAHQPS02Fmq+kTMeUcDOztf+wO3O39mtFRmwJjkggrtNatY/+zNoPEtreId96bwsAvwbT0i6X3cFrXZWIIxiaU1QIjIcEL7T98A/LST008EHnD2pl4oIiUiMlRV16Szjr2lpiHlLbyNC99WIxg0/mg2VW3es8pXPITtj72Q5uH7urYY3MRu8mMBwZjE0t3FdAtwOfG51G5wupFuFpE8p6wc+CrqnFVOWUYLT11129PBdE3JIWfi8ReBN4fSg6Yz4vw7aBmxX8rBIcwWtRmTmrS1IETkOGCtqi4WkcOiDl0J/A/IBe4Cfg5cRyirQay4OT0iMgOYATBy5MgernVqUl34BqEuDFvXkBoBmtd8TE5uPt6t4ruLvP7BbH3cZZSPHE1w8Hbd3lvbBqKNSU06u5gmAieIyDFAPlAkIg+p6pnO8RYRuQ/4mfN6FRD9VBgOrI69qareRSiwUFFRkfYnb2wwmDR2CE8uru6QUO/SOUv4yZwllMcEi8qqaht3SFGgsZ66Vx9k07J5HHLwwXxx4OWuLQP/Dvvw7o3Hsv0V3dse3QaijUld2gKEql5JqLWA04L4maqeGR5XkND//qnAe84lc4Efi8ijhAan6/t6/MEtu+rDC7+Ma9aEX0cPggIdvjfuNBhg09J51L36IMHmjQC89tprjNzmINjpkLjzSwtCezAkmz4sQmQ9SWGuF5/XE7dq3RjTub5YB/GwiAwh1KOwhNCOdQDPAccAnxDKGHtOH9StA7dV0J01WZraAlwz9302NrdnTd6kvBwPLe1dH2Npqf6AmhfuoPXrT+OObXzlPkp32J+AJzdS5vMKVx8fSt09a8oYfjInwU5vCitvPLbL9THGdNQrAUJVXwZedr53zWPgzF76UW/UJ1XdHczsbt94pupqcAg01FL78t9oeO9F1+NDR27PX++8jeZtxyUc65k6oZxrn36fWpe9GoRQ689aCsZsGVtJnUSqq6BNajQYYOO7z1D32sNoa2PccfHlUXzQaYw68jS+/e1vA8k34bn6+N25dM4S1y6/8FRWY0z3pX0ldSazVdA9p/nL5ay572JqX7zbNTgUjD2EYeffSfEBp/C/TantmDd1QnnCLj+bymrMlrMWRCfycjy2xecWaN/4DbUv3UfjB6+4HvdtNZLSyT/AP2qvSFlXpqEmSt1tU1mN2XIWIGKEp7VW1zWF9lXt6wpluNavP3MNDpLrp+TgM9hm/xNpDm6eztrVaaizpoxx3Z/BprIas+WsiylKeFpr+BOpBYctV77nRAp32rdDWeHukyi/4C52nXw6N54ygfISP0KoNdDVjXemTijntyeN26J7GGPc2Y5yURJtImO6z+/zMmlYgDsumkrOViMomzyT/OG72y5sxvQh21GuG2xgs3u0vZUN/6mkcNyR5Awq63CsqS3A0no/N/71KSpX5bNmQ6stWDMmQ1iAiGLTWruu8dP/UDv/Ltrr1tC2/iu2Pu6yuHNW1zVx+RXHc3kf1M8Y0302BhFl1pQxrhkDTby22jWsfeJa1j1xLe11oYwoDe+/RPNX78WdazOKjMlMFiCiTJ1QzhkH9E2G2EwRbGum7rWHWH3vD2n69D9xx+vfeCSuzGYUGZOZrIuJjhlbSwp8eIjfwCLbqSpNH79FzYv3ENiwNv4E8TB4wjEUH3Jmh+KJO5bZWIMxGSrrA8QvK5d3yNDqltsn27WtX0XN/DtpXlnlejyvfDfKJs8kd9sdOpSfecBIrp86rjeqaIxJg6wOEJVV1a7pu01IsLWJ+jfnsOE/lRBsjzvuKSyh9LBzKdx9kuveDRWjyuLKjDGZI6sDxOx5Kyw4JND40ZvUvHAngU3r4w+Kh8EVJ1Ay8bt48goS3sMS5hmT2bI6QNi6h8TaN653DQ75I/ek9MgfkDtkVKf3sPfXmMyW1QHC1j0kNnjCMWxaOo+2dSsB8A7aih2P/yH+MROpa4rvbnJj01uNyWxZPc3Vpl8mJh4vZUddCN4cig44mWEX3EHLyP1paAng83a+WsQS5hmT+dIeIETEKyJVIvJMTPmfRWRT1Os8EZkjIp+IyNsiMjrddct2rV9/yrrKGwm2NUfKhM37PucP353hM++j9Fvfx5Mbag20BZXC3JwOyfFumT6eW6aPt4R5xgwwvdHFdAnwAVAULhCRCqAk5rzzgFpV3UlETgN+B0xPZ8Wuffr9lM4TCe1zPFAEmjZS99qDbFryb9AgvrJySg79XiSBHhBJoe0dVBp3fX1TG0uuPiqu3AKCMQNLWlsQIjIcOBa4J6rMC8yGuNQ8JwL3O98/ARwhbnMne0hlVXVKax68IgMmOGgwwMYl/2b13T9gU9VzoKHlgPXvPElb7erIp/7Z81Yk3STJxhaMyQ7pbkHcQigQDI4q+zEwV1XXxDz/y4GvAFS1XUTqga2Ab9JRsWvmptZ6CAyQ6NCyegU1L9xB6/8+jj8YaKfxw9f5yZxhkc2SErGxBWOyR9oChIgcB6xV1cUicphTNgw4BTjM7RKXsrins4jMAGYAjBzZvbxJlVXV1DVlx4rpQGM9da/cz6Zlz7sezykZStmRM/DvGNrUJ9lOel4RG1swJoukswUxEThBRI4B8gmNQbwPtACfOK2HAhH5RFV3AlYBI4BVIpIDFAM1sTdV1buAuyC0YVB3KjZ73oruXJZRNBhgY9Vz1L/2EMGWhrjjkpNH8UHTKdp3KpKT2/Fal/vZBj/GZJ+0BQhVvRK4EsBpQfxMVY+LPkdENjnBAWAucDbwFnAysEDTsN1dZVV10i6UgbAPdfOq96l54Q7a1n7uerxgzERKDz+PnKJtUrpfid/HNSfsbsHBmCzTnxbK3Qs8KCKfEGo5nNbTP6CyqprLHl+a9JxMDg6qyvp//YmG5fNdj+eUDads8kz8o8d36b6FeTkWHIzJQr0SIFT1ZeBll/JBUd83ExqfSJvZ81YQCGZyCEhORPDmD44vz/VTMvF0Bu9zPOL1dfm+ljLDmOzUn1oQadfTaTW8Iv1ullPxxNNp+O/LBBpqASjc7TBKDjuHnMFbJbymxO+jMC8n4ftj01qNyU5ZFSB6+oHel8FBVV1TbHvyCiiZdC4b3n6SsskzyR+xR6f3Co8vVFZVRxbIhdm0VmOyV1YFiJ5+oBfmesnN8fTqJkPa3saG//yDppVVbHvaDYjEr3Us3O0wCnc9FPF4O71faYEvMr4Q/jO8u96wEj+zpoyx8QdjslRWBYjyHs7e2tAaYPyIYt5ZWUtbIP2tiaZPF1Hz4p20164BYNOyFxi815S480QEpPPg4PMIVx+/e4eyqRPKLSAYY4Asy+Y6a8oY/L7OH5xdsfCzWmafvFckwV06tNX9j7VPXc/aJ66JBAeAulfuZ5B2P+ANyrfZScaYxLKqBRHbhVJS4EOVLVpVHVCN3PcX/1hOQ2viHEZdFWxrYcPbT1C/8AkIxNdRUDb+byUydNdu3b/O9t82xiSRVQEC3LtQtr/i2W6vf/CKUFlVzawnlvZYN5Oq0vTJ29S8eDeB+q9dzhBK9jmG8iO+zyZxn2FUXuJn0tghvPThOpudZIzplqwLEG62ZGe50/cfwbVPv99jwaGtppqa+XfR/Pli1+N5w8ay9VEX8v0TJ/Hwwi9dzxHgjSsOj7y22UnGmO6wAEFobCL2AZqKiTuWcf3UcTyU4EENqU+tDbY2U//Wo2x4pxKC8Vt6egpKKD3sHAr3mISIhycXV1Ps97l2j8W2DGx2kjGmOyxA0PEBmkpLorwLD9g/nrpXp8En0LSBNfddTGCjS2Zz8VC0z/EUH/xdPHmFkeKmtgD5Pg9+nzelloHNTjLGdFVWzWJKZuqE8g7dMm78Pi+3TB/PG1cc3uFhW+JPPoPptyeNozxJf7/XX0TesPiHetH2e7Fs6RLKjrigQ3AIq21sI9/nocTvs60+jTE9zloQhProw90vibYX9QgJH77XnLA7P5mzxPXes+et6BBQflm5nIcXfhk3KF56+Hk0fboIbW/BO6iM0knnUbjroYwbN45hJYkHmmsb2/D7vNw8fbwFBmNMj8r6FkR4ALe6rgnFPTj4vMJNpyZ+ACd7MEcnulNVrjhye26ePp7yEn/kU39pgY+com0oPvh0ivafxrAL7qRwt29RXloAdL5+o6ktkBV7XBhjelfWtyAS7b/sFSGomvKAbqJV2uEB46VLl/LjH/+Y7bbbjscff7zD/cJBiv1PjpRFjyWkMkZiGVeNMT0t6wNEogdrUJXPbzw25fu4zYTy+7xceOC2XHTRRdx2220Eg0EA5s+fz5FHHhk5L5VZRuFB5ok3LkgaiIwxpqdkfYBItAaiqw/c2If80KI89m5ZwqUnf49vvuk4O+miiy5i6dKl5Obmdrg+lTGERIHI1jQYY3pa1o9BuPXvd/eBG54J9fjJ2xL45y/4v+tmxQUHgJ122okNGzZ0q75TJ5RHZkXZzCVjTDqlvQUhIl5gEVCtqseJyL1ABaEFvx8B31fVTSKSBzwA7AOsB6ar6sp016+7i8iiZz6Frzl4RB5XXXUV99xzD27bae+www786U9/4rjjjnO5Y9fqbAHBGJNu4vYg69EfIPJTQgGhyAkQRaq6wTl2E7BWVW8UkR8Ce6rqTBE5DfiOqk5Pdu+KigpdtGhRWuvvJjZ1hQYDtLz3PA1vPMSmDfVx5/v9fq666ip+9rOfkZ+f39vVNcaYDkRksapWdHZeWlsQIjIcOBa4AfgpQFRwEMAPkSUBJwLXON8/AfxFRETTHcG6IXrmU/OqD6h54Xba1n7meu60adP44x//yKhRo3qzisYYs8XSPQZxC3A5EIwuFJH7gP8BY4E/O8XlwFcAqtoO1ANxGymLyAwRWSQii9atW5fGqicWnvnU/OVyvn54lmtwGDNmDPPmzeOJJ56w4GCMyUhpCxAichyh7qO4tKSqeg4wDPgACHcjxW+wTHwWblW9S1UrVLViyJAhPVnllIVnOOWN2J3coTt3OFZYWMjvfvc7li1bxlFHHdUX1TPGmB6RzhbEROAEEVkJPAocLiIPhQ+qagCYA0xzilYBIwBEJAcoBmrSWL9uC898EvFQNvlCwrHtkG+fyIoVK7j88ss7TGE1xphMlLYAoapXqupwVR0NnAYsAL4nIjtBZAzieOBD55K5wNnO9ycDC/rL+EN1dTUbN26MvI6eapo/dBdGHnUOv7n7cV79VyXl5Ta7yBgzMPT2QjkB7heRIuf7pcCFzrF7gQdF5BNCLYfTerlucVpbW7n55pv5zW9+w4UXXsjs2bMjxzpONU19xbUxxmSKtE9zTad0TnN9/vnnueiii/joo48AyMnJYdmyZey6a/f2fzbGmP4i1WmuWb+SOtYXX3zBtGnTmDJlSiQ4ALS3t3PRRRe5LoAzxpiByAKEo7m5md/85jfsuuuuPPXUU3HHS0tLmTZtmgUIY0zWyPpkfQDPPPMMl1xyCZ99Fr+eQUS44IILuOGGG9h66637oHbGGNM3sjpAfPrpp1xyySU8++yzrsf3228//vKXv7Dvvvv2cs2MMabvZWUXU2NjI7/61a/YbbfdXIPD1ltvzb333stbb71lwcEYk7WysgVx6623cv3118eVezwefvjDH3LddddRWlraBzUzxpj+IytbEBdffDEjR47sUHbwwQfz7rvv8uc//9mCgzHGkKUBoqCggJtuugmA7bbbjgcffJBXX32Vvfbaq49rZowx/UdWdjEBnHTSSdx2222cccYZFBUV9XV1jDGm38naACEiXHjhhZ2faIwxWSoru5iMMcZ0zgKEMcYYVxYgjDHGuLIAYYwxxpUFCGOMMa4sQBhjjHFlAcIYY4yrjN5RTkTWAV/0dT2ibA1809eV6IJMqq/VNX0yqb5W154xSlWHdHZSRgeI/kZEFqWyjV9/kUn1tbqmTybV1+rau6yLyRhjjCsLEMYYY1xZgOhZd/V1Bbook+prdU2fTKqv1bUX2RiEMcYYV9aCMMYY48oCRBeJiFdEqkTkGef1vSKyVESWicgTIjLIKc8TkTki8omIvC0io/tDfaPK/ywim6Je93l9Xd7bv4nI5yKyxPka75SLiNzq1HWZiOzdD+oqInKDiHwkIh+IyMX9pa4J6vta1Pu6WkQq+0t9Xep6hIi869T1dRHZySnvj7+zhzt1fU9E7heRHKe8z9/X7rAA0XWXAB9Evb5UVfdS1T2BL4EfO+XnAbWquhNwM/C73q1mRGx9EZEKoCTmvP5Q37i6ArNUdbzztcQpOxrY2fmaAdzei3UMi63r94ERwFhV3RV41CnvD3WFmPqq6iHh9xV4C3jKOdQf6hv73t4OnOHU9e/AL53yfvU7KyIe4H7gNFXdg9AarbOd8/rD+9plFiC6QESGA8cC94TLVHWDc0wAPxAe1DmR0C8LwBPAEc45vcatviLiBWYDl8ec3qf1datrEicCD2jIQqBERIamtYJREtT1QuA6VQ0CqOra/lDXJPUNHxsMHA5UOkX98b1VILztYzGwOqqu/el3diugRVU/cl6/AEyLqmuf/h50hwWIrrmF0IM1GF0oIvcB/wPGAn92isuBrwBUtR2oJ/QL1Jvc6vtjYK6qrok5t6/r6/reAjc4TfKbRSTPKYvU1bHKKestbnXdEZguIotE5F8isrNT3td1hcTvLcB3gBfDH3To+/q61fV84DkRWQV8D7jRKe9vv7PfAD6nhQ5wMqFWJfT9+9otFiBSJCLHAWtVdXHsMVU9BxhGqKk5PXyJy216bcqYW31FZBhwCpuDWIdLXMp6pb5J3tsrCQXdfYEy4OfhS1xu09d1zQOanZWzdwN/DV/icps+/T2IcTrwSPQlLuf09Xt7KXCMqg4H7gNuCl/icps+q6uGpoSeBtwsIu8AG4H28CUut+n3U0izdk/qbpgInCAixwD5QJGIPKSqZwKoakBE5gCzCP0SryL06WGVM1BVDNT0ZX2B94EW4BOnJV4gIp84fbh9Wd+k7y3Q4rTSfua8Dtc1bDibux36pK5OnZ50zvkHod+Bvq4rJHlvRWQrYD9CrYiw/vbePktoXOdt55w5wL9j6trffmcPARCRo4BdYuoa1tu/B92jqvbVxS/gMOAZQp8KdnLKBPgD8Afn9Y+AO5zvTwMe6+v6upRvivq+X9Q3uq7A0Kj39hbgRuf1scC/nPIDgHf6QV1vBM6NKv9Pf6qr2+8BMBO4P+acflHfqP9jOYS6bnZxys8DnnS+74+/s9s4f+YBLwKH96f3tatf1oLYMgLcLyJFzvdLCQ1WAtwLPCginxD6VHNa31QxZf2xvg+LyBBC7+0SQg80gOeAY4BPgEbgnL6pXgc3EqrvpcAmQv3m0D/rGnYam/vzw/pVfVW1XUQuAJ4UkSBQC5zrHO6Pv7OznO4nD3C7qi5wyvvV+5oqW0ltjDHGlQ1SG2OMcWUBwhhjjCsLEMYYY1xZgDDGGOPKAoQxxhhXFiBM1hIRFZEHo17niMg6icl829+IyMtR6RyMSRsLECabNQB7iIjfeT0ZqO6LioTTQhvTn1iAMNnuX4RWuUJMXiIRKRSRv4rIf5yc/yc65aMltJ/Cu87XQU75UBF51dm34D0RCadciN5342QR+Zvz/d9E5CYReQn4XZKf5xeRR52khXMIZQ02Ju3sU4vJdo8Cv3a6lfYklGTvEOfYL4AFqnquiJQA74jIfGAtMFlVm52srY8AFcB3gXmqeoOTVr0ghZ+/C3CkhnJ5/b8EP+8HQKOq7ikiewLv9tjf3pgkLECYrKaqyyS0E9nphNIhRDuKUEK2cJLAfGAkoSRrf5HQDncBNidk+w/wVxHxAZW6eYOjZB5X1UAnP+9Q4Nao+i7r2t/SmO6xAGEMzCWUaPEwOu4nIMA0VV0RfbKIXAN8DexFqJu2GUBVXxWRQwl1WT0oIrNV9QE6pnXOj/nZDSn8PMiA1NBm4LExCGNC3UrXqerymPJ5wEXhXcpEZIJTXgys0dDucd8DvM7xUYT2CLibUCK58L7DX4vIrhLakjI6tXasRD/vVeAMp2wPQl1hxqSdBQiT9VR1lar+yeXQbwAfsExE3nNeA9wGnC0iCwl1L4VbAYcBS0SkitBWk+F7XkEodfUCIHYnv1R+3u3AIKdr6XLgnS7/JY3pBsvmaowxxpW1IIwxxriyAGGMMcaVBQhjjDGuLEAYY4xxZQHCGGOMKwsQxhhjXFmAMMYY48oChDHGGFf/H5aFYpfImSQAAAABSURBVLoDjCd6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#画图观察\n",
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y_test, y_test_pred)\n",
    "ax.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('Measured')\n",
    "ax.set_ylabel('Predicted')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>12. 重复步骤7-11，选择验证集上MSE或RMSE最小的模型作为模型输出</h1>\n",
    "<h1>7. 使用不同的参数在训练集上训练出模型(使用训练集交叉验证是可选的)</h1>\n",
    "    * 考虑L2正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ridge(alpha=0.1, copy_X=True, fit_intercept=True, max_iter=None,\n",
       "   normalize=False, random_state=None, solver='auto', tol=0.001)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#加入L2正则化\n",
    "from sklearn.linear_model import Ridge\n",
    "ridge = Ridge(alpha = 0.1)\n",
    "ridge.fit(X_train_norm, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[454.21842252]\n",
      "[[-14.6857877   -2.96170729   0.40929435  -2.31128706]]\n"
     ]
    }
   ],
   "source": [
    "print (ridge.intercept_)\n",
    "print (ridge.coef_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>8. 观察训练集的MSE和RMSE</h1>\n",
    "    *  考虑L2正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE: 20.999745748547557\n",
      "Train RMSE: 4.582547953764102\n"
     ]
    }
   ],
   "source": [
    "y_train_pred_ridge = ridge.predict(X_train_norm)\n",
    "from sklearn import metrics\n",
    "# 用scikit-learn计算MSE\n",
    "print (\"Train MSE:\",metrics.mean_squared_error(y_train, y_train_pred_ridge))\n",
    "# 用scikit-learn计算RMSE\n",
    "print (\"Train RMSE:\",np.sqrt(metrics.mean_squared_error(y_train, y_train_pred_ridge)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>9. 验证集数据的标准化(之前做过，此处忽略)</h1>\n",
    "<h1>10. 使用训练出的模型在验证集上做预测，并观察验证集的MSE和RMSE</h1>\n",
    "    *  考虑L2正则化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train MSE: 86.3891373545495\n",
      "Train RMSE: 9.294575695240182\n"
     ]
    }
   ],
   "source": [
    "#模型测试集L2正则化验证，比较最终表现\n",
    "y_test_pred_ridge = ridge.predict(X_test_norm)\n",
    "\n",
    "# 用scikit-learn计算MSE\n",
    "print (\"Train MSE:\",metrics.mean_squared_error(y_test, y_test_pred_ridge))\n",
    "# 用scikit-learn计算RMSE\n",
    "print (\"Train RMSE:\",np.sqrt(metrics.mean_squared_error(y_test, y_test_pred_ridge)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>11. 画图观察(可选)</h1>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOX1+PHPmckkmQSyISqEzRU3FDQqilpBwbpTUdFq3fe61LZYtYtL9VdaWrXWr/tStyp1aYpLXRD3igqGRasoWhQCFSQLkD2T8/tj7gyTmTuTScgkmcx5v155MfPcO3cehnDPPNt5RFUxxhhjonl6uwLGGGP6JgsQxhhjXFmAMMYY48oChDHGGFcWIIwxxriyAGGMMcaVBQhjjDGuLEAYY4xxZQHCGGOMq6zersCW2GqrrXTUqFG9XQ1jjEkrCxcu/E5VB3d0XloHiFGjRrFgwYLeroYxxqQVEfk6mfOsi8kYY4wrCxDGGGNcWYAwxhjjygKEMcYYVxYgjDHGuErrWUzGGNObyisqmfXyMlbXNDC0yM+MI0YzdVxpb1er21iAMMaYLiivqOSaZ5fS0BIAoLKmgWueXQrQb4KEdTEZY0wXzHp5WTg4hDS0BJj18rJeqlH3swBhjDFdsLqmoVPl6SilAUJEVojIUhFZJCILnLISEXlVRL5w/ix2ykVEbheR5SKyRET2TmXdjDFmSwwt8neqPB31RAtioqqOVdUy5/nVwGuquhPwmvMc4EhgJ+fnAuCuHqibMcZ0yYwjRuP3eduV+X1eZhwxupdq1P16o4vpeOBh5/HDwNSI8kc0aD5QJCJDeqF+xhjToanjSvndCWModVoMXpHwGER5RWUv1657pDpAKPCKiCwUkQucsm1UdQ2A8+fWTnkpsDLitaucMmOM6ZOmjisNtyQCqsDm2Uz9IUikOkBMUNW9CXYf/VhEDklwrriUacxJIheIyAIRWbBu3bruqqcxxnRJf57NlNIAoaqrnT/XAv8A9gO+DXUdOX+udU5fBQyPePkwYLXLNe9V1TJVLRs8uMN05sYYk1L9eTZTygKEiOSLyMDQY2AK8DEwBzjTOe1M4J/O4znAGc5spvFAbagryhhj+qr+PJsplS2IbYB3RGQx8AHwgqq+BMwEJovIF8Bk5znAi8BXwHLgPuCSFNbNGGO6RX+ezZSyVBuq+hWwl0v5euAwl3IFfpyq+hhjTCqE0mr0x5xMlovJGGO20NRxpR0GhHRM7GcBwhhjUqi8opIbnvuE6vqWcFm6JPazAGGMMZ0U2Roo9PsQgZr6lpiWQXTG10ihqbAWIIwxpp+IvunXNLRvGVw5exE/mb0Ir0h48Vw8XZkK25NdVRYgjDEZqys3W7eFcZFCIaGj4BA6d8LMeXHfN7p+owb5+feXVeH3SHVXlQUIY0zGiO4aqmtupSXQPkUGJL7ZdvcCuHjv67Yh0X+/+JS6/7xJ0SFnIBJMPpHKrirbD8IYkxFCN9zKmgaUYNdQKDiEJJMiIxUL4NzeN7Kl0ta4iaq597DmocvZMP8pGr6Y3+7cVK3athaEMSYjdNQ1FFJZ09Cu2ye6m2fiLoN5ZmFlUtfqjOib/OqaBlTbqPt4HtVv/JW2+prwsarX7iN3u73x+HKA1K3atgBhjOnXQjf4yk58yw51+yz4uqpdMKisaeCZhZVM26eUx+Z/k/T1BJfMo1FyfR4mzJwXDkS+6hV88/wdNK3+LObcwIa11H/2DgPGHIZAylZtW4AwxvQLbgPOQNxpph1paAnwxPsrYwabG1oCvP7ZOk4fPyJhkBABVSgt8icVnBpa2qisaSDQsIElL/8fmxa9hFtY8RZuQ8lh5+PfcX8EOG38CJvFZIwxbhItRMv1eRIGB59HGJCb1e61keLNRFpd08BNU8cAuAYRgNwsL787YQxTx5UyYea8DoOEtgXYtOQVat58hLbGjTHHJSubgv1PpGD/aXh8OZT2wGpsG6Q2xqSt8opKZjy12PUG39ASiHvjh2C3z6yT9qLiN1PCu8JF87jtUsPmPv+bpo7hTyfv5bqZTeTA84wjRuOLdzGgqfIz/vfoz6h6+f9cg4N/p/EMOfdOig76IR5fDgK8e/WklC+ysxaEMSZtXT/nE1raOl5v4CbyVTOOGO3aFeV2aa9HqGtqZdTVL3S4GK7dwLNLfNDWZta/cid1S+e6vj6reCglh1+If/t92pX3VCpxCxDGmD4n3gK26PLIVcxdEVo/EJmRtbKmIeGgcqBNw+/b0WK40I181svLYqbUAuD10Vr7bUyx+HIomXAKA8umol5fu2M+j/RYKnELEMaYXpPMwHJlTQMznl7MNc8uoaGlLfzazsxKiife+oGutUnai5xdFO99RISSwy9kzUOXgwb/bnm7HMyux1/Cr6YfDMAv/7GUuuZA+JrT9xveY/mbRJNYDt5XlZWV6YIFC3q7GsaYKMmksHBLZOf3ecn1eRKOHXSn4jwfedlZHbYauiLbK+TnZFFT34Kng66oqtfuo2lFBb+8aRa/ufDkcHm8zyg0+N1VIrJQVcs6PM8ChDGmO7nd1ELTMUMzf4CkZvb0BxpoZePCOQQ2VVM86VzXcwqzAvz6uD05cb9R7crjfUalRX7evXpSl+uUbIBIeReTiHiBBUClqh4jIpOAPwLZwELgXFVtlWBikT8DRwH1wFmq+lGq62eM6V5uK5YVeHz+N5SNLAl/801Veoi+pGHFIqrn3kPL+pWAkLfb98jZdsdwayV6qmp0yyteAO2pz64nxiCuAD4FCkTEAzwMHKaqn4vIjcCZwAPAkcBOzs/+wF3On6aPScedsUzPSdSv/7O/LwaCSekS3QDTXeuGtVTPe5D6Ze9ElCpVr97FtqfPwp/t43cnBFtTs15expWzF7kmD4zX7dVTs5hSug5CRIYBRwP3O0WDgCZV/dx5/iowzXl8PPCIBs0HikRkSCrrZzovOuFZaEFSeUVlb1fN9BGJbl4B1fDvy4wjRuP3edsdj36ebrS1hdr3/s7q+y+OCg5BretX0VoVTN1xw3OfdJg8UImdHev3eXtsFlOqF8rdBlwFhKYefAf4RCTU93UiMNx5XAqsjHjtKqfM9CFu3QfJZMA0mWPGEaNdF46FRKan/t0JYyjO2zyNs7G1exPg9aSGLz9k9YOXUPPWI2hLU8zxAXtOYej59+AbFLzlVde3JJUCJNQVJc6fWzpA3Rkp62ISkWOAtaq6UEQOBVBVFZFTgFtFJAd4BWgNvcTlMjGtKxG5ALgAYMSIEamoukkgXvdBJvQnm8Qiux7zsr3hqZluIruWGiOmrqbLnJnI/EotNf+j+rV7aVj+geu52dvuRMnki8gZ2rVv/Vs6IL0lUjkGMQE4TkSOAnIJjkE8pqqnAwcDiMgUYGfn/FVsbk0ADANWR19UVe8F7oXgLKbUVd+4iddv3FN9oqZvip65VNccwOcV98VhEa9JNgV3X1Lk9/Hu1ZP4xewPuef2W6l9/2kIxE7L9fgLKDrkDAbsORnxdK3rrCe7k9ykrItJVa9R1WGqOgo4BZinqqeLyNYATgviF8DdzkvmAGdI0HigVlXXpKp+pmvi9Rv35i+x6X1uN/pEwQFgxlOLu3WQ2tdDmeVqGlrY+2cP89efTqP230/EBgfxMGDc0Qw9/x4Gjv1+p4KDzyMU5/l6pTvJTW+spJ7hdD95gLtUdZ5T/iLBKa7LCU5zPbsX6mY6EJmSwGYx9T3dMcOsK9foShdjV3MohZQW+dvVMTqjayqtl0KqGttiynOG7kLJlIvJ3maHTl+zJ7KzdpYtlDOmn+iOVbddvca4G1/psZszbO6XjwxmPX0na/hqIWufug4AT34RxYeeTf7uEwnO5u+cnh5n6DML5YwxPSPRDLNkA0RXr9GT3zNDXZpuway7qSpom2s3kX/7fcgbfRDegYOCabhz8rv0Hn25i9YChDH9RHesuu3KLLXyisotzqraGXuPKGTquFLG3fhKSoND87oVVM29B/92e1M4/iTXc7Y6/qqELYbSIj/1za1xW1d9sVspkgUIY1Kgp1ebl1dUdsuq287OUgt9i+9J735ZxairX0jZ9dua6qh5529sXPgcaBvNaz4nf7eJZBVsFXNuvOBwekTeqVQl3OsJFiCM6WbRN4TQanMgZTeEWS8vcw0Ond3QfuIug3l8/jftriUE/w4TZs5j4i6Def2zdeHAV9/cmnbTVONRbaPuk9epfuMh2upqNpe3NFH9+gMMPv4XHV7DLSlhOk/ssABhTDfrjrGAzkqU/yiZ93Tb1znyGhAMEo/N/yZc3p/yKDV/+xVVr95FU+WnrsfbmurQ1mYkK9v1uEDCG3/kpkTpxAKEMd2sN1abx+sairfXcqSeGOztqwING6l5+zE2LfpXeMOeSN6CrSk57Dz8Ox1AMOF0rK7MQEqXhJcWIIzpZr2x2txtT+VEs2Mib1AdbWbTH6m2sWnxK9S89QhtDRtiT/D6KNz/RArGT8Pjyw13HT2zsDLpzzie3uiC7CoLEMZ0s87erLtDZ/q5o29QmRYcmlYvo2ru3TSv+cL1uH/H/Sg+7AJ8RdsCwdXNs07ai6njSikbWbLF3/x7owuyqyxAGNPNemtQMtl+7nTMf9RdNn3yOuufvwW3+V5ZRUMoPvwC8nbYF6/TqoqehtodYwnplPDSAoQxKdCTg5LR/dnRM42ig1NfvBH1FP/2ZXj8A9t1K0lWDoUHTqdg36lk5+Qw68S9Uvpvl04JL3sovZUxJhXcNnB6bP43CTd06os3op7i9Q+k6JAzws/zRk9g6Pl3UXjAyUhWNvnZWSkP7OmU8NIChDFpLJnuougNndxuUP1NoL6WeHnmBuw5mbzdvsfWJ/+WwVOvIatg6/Cx2h5YER7aKKm3NgHqDOtiMiaNJdtdFHle6Eb0s78v7ncD1BpoZeNHz1PzzuMMOuJS8nf7Xsw54vEy+NgZrq/vqdZVuqyLsABhTBqL15/tdl6k0M2pP61/aPx6CVVz76blu+BivurXH8C/4354spO76aeymydd1j1Esy4mY9JYMt1F8W58oa4Ob5wFYOmidcN3rPvn7/n2yWvDwQEgsKmK2n/Pjvu6CTuU9Eg3j9s4UfS4UF9lLQhj0lhH3UVekfCNL95sp3TtZtLWFjYsKKf230+iLU0xxyUnn6zCrSkt8jNqkJ/5X1UTUMUrwqn7D2+XLymV0mndQzQLEMakiXjdFFPHlXLl7EWur2lTZeq4Uk677z3e/bIqXB6dVyndNHy1kKrX7qW1yv1beP6Ywxn1/fP4+A/Te7hmsdJp3UO0lAcIEfECC4BKVT1GRA4DZhHs3toEnKWqy509qh8B9gHWA9NVdUWq62dMOugoPUO8sQiFlKbG7mmttd9S9dp9NHwx3/V49jY7UDL5YnJKd6Guh+sWTzqte4jWE2MQVwCRKRLvAk5T1bHA34BfOeXnAtWquiNwK/D7HqibMWkhXjfFz/6+mO2ufoH65lZ8nvQeS0ikraWJmnefYPX9F7sGB0/uQEqO+DHbnnELOaW7ACBCn+jnT6d1D9FS2oIQkWHA0cDNwE+dYgUKnMeFwGrn8fHA9c7jp4E7REQ0nTfNNqabxOuOCI0fVNe34PP23wBRPfceNi15xeWIMGDsERQdcgZef0G7I21Kn0iCZ/tBxHcbcBUwMKLsPOBFEWkANgDjnfJSYCWAqraKSC0wCPguxXU0ps8KjTsk8y2pJdB/v0sV7D+NTZ/Mg0BruCx7yGhKplxMzrY7xn1dXxkMTpd1D9FS1sUkIscAa1V1YdShK4GjVHUY8BBwS+glLpeJ+Y0XkQtEZIGILFi3bl231tmYviRyemSm85WUUrDvDwDw5BUy6KifsO2PZiUMDiHpMBjcV6WyBTEBOE5EjgJygQIReQHYRVXfd86ZDbzkPF4FDAdWiUgWwe6nqqhroqr3AvcClJWV9d+vTKbfc5uVBGTsPg2qSvOaz8kZ6t43X3jAdBChcL8T8OQOCJcX+X3UNbfGbUGlw2BwX5WyAKGq1wDXAIjIocDPganA/0RkZ1X9HJjM5gHsOcCZwHvAicA8G38w/ZXbrKQZTy0G2dxVlEnBofm7b6ieew+NXy9mm9P+QO6w3WLO8WTnUhyRaK84z0fFb6YAwc/z+jmfUBOVSyldBoP7qh5dB+GMLZwPPCMibUA1cI5z+AHgURFZTrDlcEpP1s2YnhBqNbh1G7W0pWdA8EhwQLgr2prqqX33CTYsnANtwWBZ9erdDDnzVsQTf4W4zyNcd+zu4eehPv50TWnRV0k6f0kvKyvTBQsW9HY1jGmnvKKSG577hOr64LfZIr+P648L3sz6U+6jLaGq1P3nDWreeIjAppieZEqmXMLAcUe5vlaAW6ePtRv/FhCRhapa1tF5tpLamG5UXlHJjKcXt+sPr2lo4ad/X4Sq2z5mmad57VdUvXo3Tav+43o8d+RYcoe7p8HweSXlG/qYzSxAGNNJ8boxyisq4+ZEStPeo24VaNxE7duPsbHiRdC2mOPegYMpPuw88nY+EHFJIFic5+O6Y3e34NCDLEAY0wnxUl4s+LqKZxZWZtTAcrJU26hbOpfqNx+mrb429gRvFoX7TaPggJPw+HJjDkfvC216jgUIY5KQaHC5oSXAE++vtODgomnNF1S9ehfNaz53Pe7fvoziwy/AVzw05ljkLCXTOyxAGJNA9IBzPBYcYqkq61+6nZa1/405llW0LcWHXUDejvu5vtbv87abpWR6h20YZEwcoe6kjoKDcScilBx+YfuyrGwKDzqNoefeGQ4OQs9t3mM6x1oQJq11Zd57sq9xy6BqOid3+B7k7fY96v/zJv6dD6Bk0nlkFW6DEJzRZeMLfZsFCJO2Otojwc2vypfy+PxvwtNNE73GcvgkJ1BXTdPqZeTtNN71ePGh5zBg90nsuPdBtoAtzViAMGmrs1s5lldUtgsOHb2mKM9n3UsJaFuAjR89T83bj6OBFoaedxe+om1jzssaOIiRw4fx7tWTeqGWZktYgDBpq7NbOSZKmx16TaLZSmazxm+WUvXq3bR893W4rPq1+9h62q9jzvV5JJwPyVJhpBcLECZtdXYrx0RdRkOL/DFdVt3B55G0zbHkpnXjd1S//hD1n74Zc6xh+fs0rFiEf9TYcFkozUhoIWFnuwRN77JZTCZtdXYrx3iBQ5xrpWJQOhQc0n2vNw20UPv+06y+7yLX4CDZeRRPOp/c4XuEy0qL/Cy6bkq7HdXidQmavskChElbU8eV8rsTxiQ9PdItoAhw2vgRTB1XmtJB6XRuQzT8t4LVD15GzRt/RVsaY47n73EYpeffQ8G+xyPeYKeEW6DubJeg6X3WxWTSWme2cuxob+B4XVZufF4hyyM0tMTmFOovWmvXUj3vfuo//7frcd/W21My+SJyh+1GcZ6PvOyshGMLne0SNL3PAoTpt+INiMYLKBN3Gew6y8mV0m+Dg7Y2U/vBs2x47ym0tSnmuCcnn6JDzmDA2O8jHi8+rySVRG/GEaNjxnhsQ5++zQKE6TciA0Jh1DaUHQ2IlldU8szCyqSCg1f618BzNA20sPGj512CgzBgrykUHXIG3rxCoHMZVjtqwZm+J+GGQSLy00QvVtVbur1GnWAbBpnIaamh1bmJlBb5Xefjj73hlZjtKt34fd6MWF296eN5rH9h83/v7CE7UzL5InKG7AzA6eNHcNNU9z0bTN+X7IZBHQ1SD3R+yoCLgVLn5yIgdtNY94p4RaRCRJ53nr8tIoucn9UiUu6Ui4jcLiLLRWSJiOydzPVN5gpNmwz1ayfznd5tQLS8ojKp4OAVYdo+mfFtN3/3ieSU7obHX0DJ9y9n2x/9MRwcivw+ykaW9HINTU9I2MWkqjcAiMgrwN6qutF5fj3wVJLvcQXwKVDgXPPg0AEReQb4p/P0SGAn52d/4C7nT2NcdWVaauSAaGcXxQVUeWZhZafer69SVRq+eA9VJX/0hJjjIsKgY36KJycfr39gu9ZZTUOLrV/IEMlOcx0BNEc8bwZGdfQiERkGHA3c73JsIDAJKHeKjgce0aD5QJGIDEmyfiYDdXZ6pBAciIbY1key+kP3Usv6Vaz9+29Y94//R9Urd9LWuMn1PF/RtsHgILGtM1u/kBmSHaR+FPhARP5B8HflB8AjSbzuNuAqgt1U0X4AvKaqG5znpcDKiOOrnLI1kS8SkQuACwBGjBiRZPVNf9TRtFQPEDnPSIFnFlZSNrIkIzO1tjXVU/vebDZ8+E9oaw2W1ddS887jMWm5Q3xeabe/diRbv9D/JdWCUNWbgbOBaqAGOFtV/1+i14jIMcBaVV0Y55RTgSciX+L21i51uVdVy1S1bPDgwclU3/RD5RWVVNXFTsEM/RJ5RXCbhBr65ptJNzdVpe4/b7L6/ovY8P4z4eAQsnHRSwTqasLPvSLhhYf52fG/Q9r6hf6vM9Nc84ANqvqQiAwWke1UNXarqM0mAMeJyFFALlAgIo+p6ukiMgjYj2ArImQVMDzi+TBgdSfqZzJEeUUlM55a7DrVNDvLQ1NrW8Id3jIpEV/zuhVUvXo3TSs/dj2eM2JPSg6/EG9+UbisTZX/zjwagO2ufiHutW39Qv+XVIAQkesIzmQaDTwE+IDHCAYBV6p6DXCN8/pDgZ+r6unO4ZOA51U1ct3+HOBSEXmS4OB0raq2614ymS2ZQeWm1v65eK2z2prqqHn7cTZ+9Dxo7GeSUziYkUddROPw/RBp33iPbBnE68YrzvPZAHUGSHaQ+gfAcUAdgKquxn1cIVmn0L57CeBF4CtgOXAfcMkWXN/0M10dVM40qm1sWvoalfdeyMaFc2KDgyeLgvEnMfLCezjp5JPIi+pCil7ZHC8hou0XnRmS7WJqVlUVEQUQkfzOvImqvgG8EfH8UJdzFPhxZ65rMkcmDip3Vsv6Vaz/159pqvzU9XjudntTcviF+EpKaQJe/2wdvzthTMKVzbb6ObMlGyD+LiL3EJx6ej5wDi5TV43pLtF5lKzl0DHJ8tH87Zcx5d7CbSg57Hz8O+7frjtpdU1DUskOO5MQ0fQvSQUIVf2jiEwGNhAch/iNqr6a0pqZjNJRHqVk0mjE4/NAP82r105W4TYUjD+J2nceDxZ4fRSOP5GC/U/E48uJOd9mIZmOJDtI/XtV/QXwqkuZMVskeqcxt7QXCl0OEpk0bl24/zTqPn4N31YjKT7sfNc9osGyqJrkJDtIPdml7MjurIjJXMmOL4SCRGf1p7yrgboavnvxzzSsWOR6XLKy2faMW9h62q/bBYciv4/iPF9SGysZE5KwBSEiFxOcTbSDiCyJODQQcN9FxJhOSnbRmlck4fqG/kzbAmyseJHadx6nrXETzas/I/fsv4R3cIPNn4/XX9DutfEy2BrTkY66mP4G/Av4HXB1RPlGVa1KWa1MRklmEDpT0my7aVz5MVWv3k3LuhXhspb1K9m48DkK9tu81jSgGvM5WVeS2RIJu5hUtVZVVwB/BqpU9WtV/RpoERHLtGq6xYwjRuPzxO88EmDaPqWUZtigauumKr577o98+7er2wWHkLrP3iFyP5civ69Te3Qb05Fkp7neBUTuz1DnUmZMlyz4uirhDm0KPDb/G4rzfD1XqV6kgVY2LpxDzbtPoM2xLStPTh6FB57KwH2ObTdtVcSmpJrulWyAEI34qqKqbSJi25WaLVZeUcnj879J6tzq+o439Ul3DSsWUT33HlrWr3Q9Pnjs4eRMOAPvgNgNe2oy4PMxPSvZm/xXInI5wVYDBAeuv0pNlUwmmfXysn41y6irWjespXreg9Qve8f1uG/wKEomX4R/+B5xx2yKMqSFZXpOsgHiIuB24FcEW/yv4ezJYIyb6JXQE3cZzOufrYtJ19Bf0m6XFvnDf7f65tZOtXY2fTyPqlf+D22JTV/uycmn6JAfMWDskYjHG/7sZjy9OGafhk2NrZRXVFoXk+k2ye4HsVZVT1HVrVV1G1X9oaquTXXlTHqKTKynBFdCPzb/m3bPr3l2KeUVlf1iNW/02MjRew6JSXCXiK94qGtwGLDnFIaefw8D9z4G8QSvN3GXwUwdV+q6T0NLm9oub6ZbdbQO4ipV/YOI/AX3zXsuT1nNTFqIbinMOGI018/5pMMpqQ0tAa6f8wm7Dx2Y1nmWPAK19S3hFkMoGE7YoYQV6xtY7QTFRHJKdyF/zGTqlgYTFWRvuxMlky8iZ2js9NTXP1sHQK3LanOwXd5M9+qoiymUFnJBqiti0kNkQCjK87GpsTU8A6mypiHuRj5uahpaePfL9F5OE++v+u6XVRTn+bh1+liufXYJ9S1tqLYh4t5oL/7emTSt+piC/aYxYM/J4RZDtFAAiDcO0R9aZKbvSBggVPU558+He6Y6pq9waxkA/OypxQScu6JbP3uywSETVNe38NO/LyLQpjR8MZ/qN//K1if8Ct+g4THnevOLGHre3XEDQ0goAMw4YnS7/FVgi+JM9+uoi+k5EqSyUdXjur1GptdFJ88LjRkE2trCwcEkp2l9JVVz76Xxv8Gt2avm3svWJ98Ys4sb0C44eCS2dRIZAGyfBtMTOupi+qPz5wnAtgS3GQU4FViRzBuIiJdgF1Wlqh4jwf8ZNxHcdjQA3KWqtzvlfwaOAuqBs1T1o078XUw3cUuel6lpLhLxeSVmJlFIW3Mjte89yYYPyqGtNVzeuKKChs/fI2/0gZQmSDGiCrdNH9vhZj4WEEwqddTF9CaAiPxWVQ+JOPSciLyV5HtcQXAsI5RB7CxgOLCLs+Bua6f8SGAn52d/gmsuLJ1HL7CBzo7lZ3tpdskjrqrUf/YO1a8/QGDjd7EvFA8tVasAePfqSUyYOS/uWIIFANPbkk33PVhEtg89EZHtgMEdvUhEhgFH0373uYuBG1WDm+VGTJc9HnhEg+YT3L1uSJL1M93IBjo7VtcciBlvaV73Nd8++Uu+m/N71+CQM3wPhpx9O4UHnIzfF/yvF2/PZxtLMH1BsgvlrgTeEJHQ6ulRwIVJvO424CqC6cFDdgCmi8gPgHXA5ar6BVAKROYXWOWUrUmyjqYT3AahQ99W3QZATXxtTfXUvPs3Ni58DtpiPzPvgBKKJ55L3q6HhMdRgLe5AAAgAElEQVQefnfCnoCNJZi+LdktR18SkZ2AXZyiz1Q1dmVPBBE5BlirqgtF5NCIQzlAo6qWicgJwIPAwbjvBRPTwSsiF+Cs4h4xYkQy1TdR4g1CQ/t+7Z/9fXHG7r+QDFWl7pPXqXnjIQJ11bEneLwU7DuVwgOm48nJi3sd60oyfVVSXUwikgfMAC5V1cXACCcAJDIBOE5EVgBPApNE5DGCLYNnnHP+AezpPF5FcGwiZBiwOvqiqnqvqpapatngwR32chkXNzwXu5CtoSXQbhXu1HGltHUQHLqyu1t/8t1zs1j/wi2uwSF31DiGnnMHxYee7RocQivJjenLkh2DeAhoBg5wnq8iOBMpLlW9RlWHqeoo4BRgnqqeDpQDoe2tvgd87jyeA5whQeOBWlW17qVuVl5RGTdPUPTgdEdjEV3dArS/yNv5wJgyb8FgBk+9lq1PvtF1vUNIdEA2pi9KNkDsoKp/AFoAVLWBrt8bZgLTRGQpwZ3qznPKXySYIXY5cB/BjLGmmyW6KUUHhIm7dNxCy+QOqLzRE8gd6TSAvVkUHjCdoefdRd7oA13XOUSz2WKmr0t2kLpZRPw49wMR2QFIOAYRSVXfAN5wHtcQnNkUfY4CP072mv3Rr8qX8sT7K4P7Cotw6v7DuWnqmG59j0Q3pbqmzdlAyysqeeJ99z0JMk2gcRPe3AFAcNe2uqZgehERoeTwi6l962EKJ56Dr3hop65rs8VMX5dsgLgOeAkYLiKPExxfOCtVlcpEvypfymMRG+cEVMPPy0aWdGqWS3lFJdfP+YQaJ6FbcZ6P647dnanjShPu/1zT0MI1zy5lwddVPLOwMqMGqP0+L9P2KWX2ByvD01cD9bXUvPkw9V/MZ+h5d5E7oAiRYDoRrwgBVUbtuDMzfvw0s15eljDpoNC+tWVTWU06EO1oIDLYVh5GcHXzeIK/6/NV1WUVUM8qKyvTBQv6Rx7BHa550fWG7BHIyYrdiD56r+HQtNV4NymfV5i+73CeX7wmHDhMUGlE0C2vqOS68iWseu85at5+lLbGTQCU7H0kJd+/tN3K6ch/h+iZYfHex6aymr5ARBaqallH53XYglBVFZFyVd0HeKFbamdixPu23qaxaS5CA5yhG0wyN6eWgPL4/G8yeswgOjWGW6AdXL8Cz5xrqVq0qN1rqypeInuPyeQM2TlcFvnv0NHU4NIiP+9ePSmm3Ji+LNlB6vkism9Ka5LhvEkMakaKHEtwy53kJpODw+njRzDrxL0oLfIjBLvdcrI8XDl7ERNmzuOhVys488wzOeigg1gUFRwAsopLXRfBRf47TB1Xyp9O3stWRpt+I9kxiInARc6ahjqcLlVV3TPhq0zSTt1/eLsxiBC/z0NDS2zOn8gBTpsN07HXP1vHTVNju4M00Mqnrz7Bedc9Tltz7Oc4YMAApp33E97I2ge8sXs+Rw8028po058kGyCOTGktTHi2UvQsprKRJQnz/pdXVOJxBkxNfG4trsavl1A1925avosNzAA//OEP+cMf/sDJjy4DlyAs4NoysJXRpr/oaD+IXOAiYEdgKfCAqrYmeo3pupumjok7rdXtG2nom7AFh2BfaWw7a7PIb/rffLOSqtcfoP6zt13PHTNmDHfccQeHHBJMYLy6JrbLCYJddhYITH/WUQviYYKL494m2IrYjWD6bpMi8ZLoud2Ikh17yASJggNs/qb/1PtfseaRK13TY3hy8hg5+Ww27DqFa/7dyoyBwTUhhX6f68yvIn9sl5Mx/UlHAWI3VR0DICIPAB+kvkqZq6MkeqFzQgHE2g3JKfL7wi2u3zz/OQP3/QE1bzzY7pz8MYczaOJZtPmLgPaffbz5AzUNLUyYOc/GGEy/1dEspvDXJutaSr14O7mF0mOEAkilBYek+X1erj9ud2Dz51tQdixZJcMAyN5mB7Y9/Y9sddRPECc4hIQ++5o4uatgcyCxxHumP+qoBbGXiGxwHgvgd56HZjEVxH+p6ax4s5FC5dallDxtbSZQu5amrYbxk9mL2q1PEK+PQVMuoaW6kgF7Tmm3F3S0UFdfolXS0etSjOkvErYgVNWrqgXOz0BVzYp4bMGhm8XLzRMqt+ms7jwEF8GF1C9/n9UPXMK3T99AoKUZiF2ImDtyTwaOPTJhcADC40DRaxui2b+N6Y+SneZqtkCi3dsixdvJrbquie2ufsGms8bRBuR6hIbvKql67T4avvwwfGzDh+UUHnByl64bmk4cubYhXkvCEu+Z/qjDXEx9WTrkYnJLg+HzCvnZWdQ2tMQEjF+VL834lBid1dbcSO38p9jwwTMQaD9UJlk5DD3/LrIKtg6XlSboMsrP9lLfHIgbyN3+Pd1SdhjTl3VbLiazZdzGDVoCGp42WVnTwIynFoePPfH+SgsOSVJV6pe9S/W8BwhsXBd7gngYsNcUPNmbd3TzivDu1ZOYMHOea5Aoysvmkxvj50yyldImk1iASLFk+qZb2pRrn12CYl1IyWr5biVVc++h8Wv3RWw5w3anZPJFZG+9XbvyU/cP7vLW0YSARGyltMkUFiBSrKMZMCH1LvmWTKy2pnpq//0kGxb80zV5nndACcUTzyFv1+/F7Op2+vgR4ZXq8f5dbCzBmM2SzebaZSLiFZEKEXneef5XEfmviCxyfsY65SIit4vIchFZIiJ7p7puPSGZGTAmMY8Eu5M2ffI6q++/iA0fPBsbHDxeCvY7gaHn3U3+boe6bvn5+mfrwusV3P5dLOuqMe31RAviCuBTIHJa7AxVfTrqvCOBnZyf/YG7nD/TWjIzYExibQqtVatY/8KtoLEtrcId9ib/0PPxbTU84XXcVqbbWIIx8aU0QIjIMIL7T98M/LSD048HHnH2pp4vIkUiMkRV16Syjj2lqi7pLbyNC9+g4QwYeySbKjbvWeUrHMx2R19M47B9XVsMbqI3+bGAYEx8qe5iug24ithcajc73Ui3ikiOU1YKrIw4Z5VTltbKKyqZ8dRi1z0dTOcUHXw6Hn8BeLMoPnA6w8+7m6bh+yUdHEJsUZsxyUlZC0JEjgHWqupCETk04tA1wP+AbOBe4BfAjQTTd0SLmdIjIhcAFwCMGDGim2udnGQXvkGwC6OlzWYmJUOAxv8tJ8uXg3dQbHeR1z+QrY75GaUjRtE2cNsu761tA9HGJCeVXUwTgONE5CggFygQkcdU9XTneJOIPAT83Hm+Coi8KwwDVkdfVFXvJRhYKCsrS/mdNzoYTNxlMM8srGyXcfXK2Yv4yexFlEYFi/KKSht3SFKgYQM1bz3CpsUvc/BBB/H1AVe5tgz82+/DRzOPZruru7Y9ug1EG5O8lAUIVb2GYGsBpwXxc1U9PTSuIMH//VOBj52XzAEuFZEnCQ5O1/b2+INb+m23Vc6h55GDoEC7x8adtgXYtOQVat58hLbGjQC8/fbbjNj6QNjx4Jjzi/OCezAkmj4sAqHlJPnZXnxej+uqdWNMYr2xDuJxERlMsEdhEcEd6wBeBI4ClgP1wNm9ULd23FZBd9RkaWgJcP2cT9jY2Joxi95ysjw0tXZ+jKWp8lOqXr2b5m+/jDm28c2HKN5+fwKe7HCZzytcd2wwdfeMI0bzk9lxdnpTWDHz6E7XxxjTXo8ECFV9A3jDeeyax8CZvfTjnqhPsro6mNnVvvF01dngEKirofrNv1K3dK7r8SEjtuPBe+6kcZsxccd6po4r5YbnPqHaZa8GIdj6s5aCMVvGVlInkOwqaJMcbQuw8aMXqHnncbSpLua4+HIoPPAURh5+Ct///veBxHs+X3fs7lw5e5Frl5/tz2DMlkv5Sup0Zqugu0/jyo9Z89crqH7tXtfgkLfLwQw97x4Kx5/E/zYltynS1HGlcbv8bCqrMVvOWhAdyMny2C5uW6B143qq33iQ+v+86XrcN2gExZMvxD9yr3BZZ6ahxkvdbVNZjdlyFiCihKa1VtY0BPdV7e0Kpbnmb5e7BgfJ9lN00Glsvf/xNLZtns7a2Wmobpss2VRWY7qHdTFFCE1rDX0jteCw5YaOmUD+jvu2K8vffSKl59/LrpNPZeZJ4ygt8iMEWwOd3Xhn6rhSfnfCmC26hjHGne0oFyHeJjKm6/w+LxOHBrj7sqlkDRpOyeSLyB22u+3CZkwvsh3lusAGNrtGW5vZ8GE5+WMOJ2tASbtjDS0BFtf6mfngs5SvymXNhmZbsGZMmrAAEcGmtXZe/ZcfUj33Xlpr1tCyfiVbHfOzmHNW1zRw1dXHclUv1M8Y03U2BhFhxhGjXTMGmlgt1WtY+/QNrHv6BlprghlR6j55ncaVH8ecazOKjElPFiAiTB1XymnjeydDbLpoa2mk5u3HWP3AJTR8+WHM8dp3n4gpsxlFxqQn62KifcbWojwfHmI3sMh0qkrDF+9R9dr9BDasjT1BPAwcdxSFB5/ernjCDiU21mBMmsr4APGr8qXtMrS65fbJdC3rV1E19x4aV1S4Hs8p3Y2SyReRvc327cpPHz+Cm6aO6YkqGmNSIKMDRHlFpWv6bhPU1txA7b9ns+HDcmhrjTnuyS+i+NBzyN99ouveDWUjS2LKjDHpI6MDxKyXl1lwiKP+839T9eo9BDatjz0oHgaWHUfRhB/iycmLew1LmGdMesvoAGHrHuJr3bjeNTjkjtiT4sMvJHvwyA6vYZ+vMektowOErXuIb+C4o9i0+GVa1q0AwDtgEDscewn+0ROoaYjtbnJj01uNSW8ZPc3Vpl/GJx4vJVMuBm8WBeNPZOj5d9M0Yn/qmgL4vB2vFrGEecakv5QHCBHxikiFiDwfVf4XEdkU8TxHRGaLyHIReV9ERqW6bpmu+dsvWVc+k7aWxnCZsHnf59xhuzPsooco/t5ZeLKDrYGWNiU/O6tdcrzbpo/ltuljLWGeMf1MT3QxXQF8ChSECkSkDCiKOu9coFpVdxSRU4DfA9NTWbEbnvskqfNEgvsc9xeBho3UvP0omxa9BNqGr6SUokN+FE6gB4RTaHsHFMe8vrahhUXXTYkpt4BgTP+S0haEiAwDjgbujyjzArMgJjXP8cDDzuOngcPEbe5kNymvqExqzYNXpN8EB9U2Ni56idX3XcimihdBg8sBaz94hpbq1eFv/bNeXpZwkyQbWzAmM6S6BXEbwUAwMKLsUmCOqq6Juv+XAisBVLVVRGqBQcB3qajY9XOSaz0E+kl0aFq9jKq5d9O85ovYg4FW6j97h5/MHhreLCkeG1swJnOkLECIyDHAWlVdKCKHOmVDgZOAQ91e4lIWc3cWkQuACwBGjOha3qTyikpqGjJjxXSgvpaaNx9m05JXXI9nFQ2h5PAL8O8Q3NQn0U56XhEbWzAmg6SyBTEBOE5EjgJyCY5BfAI0Acud1kOeiCxX1R2BVcBwYJWIZAGFQFX0RVX1XuBeCG4Y1JWKzXp5WVdella0LcCmRf+i5q1HaWuqizkuWTkUHjidgn2nIlnZ7V/rcj3b4MeYzJOyAKGq1wDXADgtiJ+r6jGR54jIJic4AMwBzgTeA04E5mkKtrsrr6hM2IXSH/ahblz1CVWv3k3L2v+6Hs8bPYHiSeeSVbB1Utcr8vu4/rjdLTgYk2H60kK5B4BHRWQ5wZbDKd39BuUVlfzsqcUJz0nn4KCqrP/Xn6lbOtf1eFbJMEomX4R/1NhOXTc/J8uCgzEZqEcChKq+AbzhUj4g4nEjwfGJlJn18jICbekcAhITEby5A2PLs/0UTTiVgfsci3h9nb6upcwwJjP1pRZEynV3Wg2vSJ+b5VQ44VTq/vMGgbpqAPJ3O5SiQ88ma+CguK8p8vvIz8mK+/nYtFZjMlNGBYjuvqH3ZnBQVdcU256cPIomnsOG95+hZPJF5A7fo8NrhcYXyisqwwvkQmxaqzGZK6MCRHff0POzvWRneXp0kyFtbWHDh/+gYUUF25xyMyKxax3zdzuU/F0PQTzeDq9XnOcLjy+E/gztrje0yM+MI0bb+IMxGSqjAkRpN2dvrWsOMHZ4IR+sqKYlkPrWRMOXC6h67R5aq9cAsGnJqwzc64iY80QEpOPg4PMI1x27e7uyqeNKLSAYY4AMy+Y644jR+H0d3zg7Y/5X1cw6ca9wgrtUaKn5H2ufvYm1T18fDg4ANW8+zADtesAbkGuzk4wx8WVUCyK6C6Uoz4cqW7SqOqAavu4v/7GUuub4OYw6q62liQ3vP03t/KchEFtHQdn4vxXIkF27dP0a23/bGJNARgUIcO9C2e7qF7q8/sErQnlFJTOeXtxt3UyqSsPy96l67T4Ctd+6nCEU7XMUpYedxSZxn2FUWuRn4i6Def2zdTY7yRjTJRkXINxsyc5yp+4/nBue+6TbgkNLVSVVc++l8b8LXY/nDN2FraZczFnHT+Tx+d+4niPAu1dPCj+32UnGmK6wAEFwbCL6BpqMCTuUcNPUMTwW50YNyU+tbWtupPa9J9nwQTm0xW7p6ckrovjQs8nfYyIiHp5ZWEmh3+faPRbdMrDZScaYrrAAQfsbaDItidJO3GD/dPJeHQafQMMG1jx0OYGNLpnNxUPBPsdSeNAP8eTkh4sbWgLk+jz4fd6kWgY2O8kY01kZNYspkanjStt1y7jx+7zcNn0s7149qd3NtsifeAbT704YQ2mC/n6vv4CcobE39YLt9mLJ4kWUHHZ+u+AQUl3fQq7PQ5HfZ1t9GmO6nbUgCPbRh7pf4m0v6hHi3nyvP253fjJ7keu1Z728rF1A+VX5Uh6f/03MoHjxpHNp+HIB2tqEd0AJxRPPJX/XQxgzZgxDi+IPNFfXt+D3ebl1+lgLDMaYbpXxLYjQAG5lTQOKe3DweYVbTo5/A050Y45MdKeqXH34dtw6fSylRf7wt/7iPB9ZBVtTeNCpFOw/jaHn30P+bt+jtDgP6Hj9RkNLICP2uDDG9KyMb0HE23/ZK0KbatIDuvFWaYcGjBcvXsyll17Ktttuy1NPPdXueqEgxf4nhssixxKSGSOxjKvGmO6W8QEi3o21TZX/zjw66eu4zYTy+7xcfMA2XHbZZdx55520tbUBMHfuXA4//PDwecnMMgoNMk+YOS9hIDLGmO6S8QEi3hqIzt5wo2/yQwpy2LtpEVee+CO++6797KTLLruMxYsXk52d3e71yYwhxAtEtqbBGNPdMn4Mwq1/v6s33NBMqKdO3IbAP3/J/904IyY4AOy4445s2LChS/WdOq40PCvKZi4ZY1Ip5S0IEfECC4BKVT1GRB4Ayggu+P0cOEtVN4lIDvAIsA+wHpiuqitSXb+uLiKLnPkUes1Bw3O49tpruf/++3HbTnv77bfnz3/+M8ccc4zLFTtXZwsIxphUE7cbWbe+gchPCQaEAidAFKjqBufYLcBaVZ0pIpcAe6rqRSJyCvADVZ2e6NplZWW6YMGClNbfTXTqCm0L0PTxK9S9+xibNtTGnO/3+7n22mv5+c9/Tm5ubk9X1xhj2hGRhapa1tF5KW1BiMgw4GjgZuCnABHBQQA/hJcEHA9c7zx+GrhDRERTHcG6IHLmU+OqT6l69S5a1n7leu60adP405/+xMiRI3uyisYYs8VSPQZxG3AV0BZZKCIPAf8DdgH+4hSXAisBVLUVqAViNlIWkQtEZIGILFi3bl0Kqx5faOZT4zdL+fbxGa7BYfTo0bz88ss8/fTTFhyMMWkpZQFCRI4h2H0Uk5ZUVc8GhgKfAqFupNgNlonNwq2q96pqmaqWDR48uDurnLTQDKec4buTPWSndsfy8/P5/e9/z5IlS5gyZUpvVM8YY7pFKlsQE4DjRGQF8CQwSUQeCx1U1QAwG5jmFK0ChgOISBZQCFSlsH5dFpr5JOKhZPLFhGLbwd8/nmXLlnHVVVe1m8JqjDHpKGUBQlWvUdVhqjoKOAWYB/xIRHaE8BjEscBnzkvmAGc6j08E5vWV8YfKyko2btwYfh451TR3yM6MmHI2v73vKd76VzmlpTa7yBjTP/T0QjkBHhaRAufxYuBi59gDwKMispxgy+GUHq5bjObmZm699VZ++9vfcvHFFzNr1qzwsfZTTZNfcW2MMeki5dNcUymV01xfeeUVLrvsMj7//HMAsrKyWLJkCbvu2rX9n40xpq9Idpprxq+kjvb1118zbdo0jjjiiHBwAGhtbeWyyy5zXQBnjDH9kQUIR2NjI7/97W/ZddddefbZZ2OOFxcXM23aNAsQxpiMkfHJ+gCef/55rrjiCr76KnY9g4hw/vnnc/PNN7PVVlv1Qu2MMaZ3ZHSA+PLLL7niiit44YUXXI/vt99+3HHHHey77749XDNjjOl9GdnFVF9fz69//Wt222031+Cw1VZb8cADD/Dee+9ZcDDGZKyMbEHcfvvt3HTTTTHlHo+HSy65hBtvvJHi4uJeqJkxxvQdGdmCuPzyyxkxYkS7soMOOoiPPvqIv/zlLxYcjDGGDA0QeXl53HLLLQBsu+22PProo7z11lvstddevVwzY4zpOzKyiwnghBNO4M477+S0006joKCgt6tjjDF9TsYGCBHh4osv7vhEY4zJUBnZxWSMMaZjFiCMMca4sgBhjDHGlQUIY4wxrixAGGOMcWUBwhhjjCsLEMYYY1yl9Y5yIrIO+Lq36xFhK+C73q5EJ6RTfa2uqZNO9bW6do+Rqjq4o5PSOkD0NSKyIJlt/PqKdKqv1TV10qm+VteeZV1MxhhjXFmAMMYY48oCRPe6t7cr0EnpVF+ra+qkU32trj3IxiCMMca4shaEMcYYVxYgOklEvCJSISLPO88fEJHFIrJERJ4WkQFOeY6IzBaR5SLyvoiM6gv1jSj/i4hsinje6/V1+Wz/KiL/FZFFzs9Yp1xE5HanrktEZO8+UFcRkZtF5HMR+VRELu8rdY1T37cjPtfVIlLeV+rrUtfDROQjp67viMiOTnlf/J2d5NT1YxF5WESynPJe/1y7wgJE510BfBrx/EpV3UtV9wS+AS51ys8FqlV1R+BW4Pc9W82w6PoiImVAUdR5faG+MXUFZqjqWOdnkVN2JLCT83MBcFcP1jEkuq5nAcOBXVR1V+BJp7wv1BWi6quqB4c+V+A94FnnUF+ob/RnexdwmlPXvwG/csr71O+siHiAh4FTVHUPgmu0znTO6wufa6dZgOgEERkGHA3cHypT1Q3OMQH8QGhQ53iCvywATwOHOef0GLf6iogXmAVcFXV6r9bXra4JHA88okHzgSIRGZLSCkaIU9eLgRtVtQ1AVdf2hbomqG/o2EBgElDuFPXFz1aB0LaPhcDqiLr2pd/ZQUCTqn7uPH8VmBZR1179PegKCxCdcxvBG2tbZKGIPAT8D9gF+ItTXAqsBFDVVqCW4C9QT3Kr76XAHFVdE3Vub9fX9bMFbnaa5LeKSI5TFq6rY5VT1lPc6roDMF1EFojIv0RkJ6e8t+sK8T9bgB8Ar4W+6ND79XWr63nAiyKyCvgRMNMp72u/s98BPqeFDnAiwVYl9P7n2iUWIJIkIscAa1V1YfQxVT0bGEqwqTk99BKXy/TYlDG3+orIUOAkNgexdi9xKeuR+ib4bK8hGHT3BUqAX4Re4nKZ3q5rDtDorJy9D3gw9BKXy/Tq70GUU4EnIl/ick5vf7ZXAkep6jDgIeCW0EtcLtNrddXglNBTgFtF5ANgI9AaeonLZfr8FNKM3ZO6CyYAx4nIUUAuUCAij6nq6QCqGhCR2cAMgr/Eqwh+e1jlDFQVAlW9WV/gE6AJWO60xPNEZLnTh9ub9U342QJNTivt587zUF1DhrG526FX6urU6RnnnH8Q/B3o7bpCgs9WRAYB+xFsRYT0tc/2BYLjOu8758wGXoqqa1/7nT0YQESmADtH1TWkp38PukZV7aeTP8ChwPMEvxXs6JQJ8Efgj87zHwN3O49PAf7e2/V1Kd8U8bhP1DeyrsCQiM/2NmCm8/xo4F9O+Xjggz5Q15nAORHlH/alurr9HgAXAQ9HndMn6hvxfyyLYNfNzk75ucAzzuO++Du7tfNnDvAaMKkvfa6d/bEWxJYR4GERKXAeLyY4WAnwAPCoiCwn+K3mlN6pYtL6Yn0fF5HBBD/bRQRvaAAvAkcBy4F64OzeqV47MwnW90pgE8F+c+ibdQ05hc39+SF9qr6q2ioi5wPPiEgbUA2c4xzui7+zM5zuJw9wl6rOc8r71OeaLFtJbYwxxpUNUhtjjHFlAcIYY4wrCxDGGGNcWYAwxhjjygKEMcYYVxYgTMYSERWRRyOeZ4nIOonKfNvXiMgbEekcjEkZCxAmk9UBe4iI33k+GajsjYqE0kIb05dYgDCZ7l8EV7lCVF4iEckXkQdF5EMn5//xTvkoCe6n8JHzc6BTPkRE3nL2LfhYREIpFyL33ThRRP7qPP6riNwiIq8Dv0/wfn4RedJJWjibYNZgY1LOvrWYTPck8BunW2lPgkn2DnaO/RKYp6rniEgR8IGIzAXWApNVtdHJ2voEUAb8EHhZVW920qrnJfH+OwOHazCX1/+L834XAvWquqeI7Al81G1/e2MSsABhMpqqLpHgTmSnEkyHEGkKwYRsoSSBucAIgknW7pDgDncBNidk+xB4UER8QLlu3uAokadUNdDB+x0C3B5R3yWd+1sa0zUWIIyBOQQTLR5K+/0EBJimqssiTxaR64Fvgb0IdtM2AqjqWyJyCMEuq0dFZJaqPkL7tM65Ue9dl8T7QRqkhjb9j41BGBPsVrpRVZdGlb8MXBbapUxExjnlhcAaDe4e9yPA6xwfSXCPgPsIJpIL7Tv8rYjsKsEtKSNTa0eL935vAac5ZXsQ7AozJuUsQJiMp6qrVPXPLod+C/iAJSLysfMc4E7gTBGZT7B7KdQKOBRYJCIVBLeaDF3zaoKpq+cB0Tv5JfN+dwEDnK6lq4APOv2XNKYLLJurMcYYV9aCMMYY48oChDHGGFcWIIwxxriyAGGMMcaVBQhjjDGuLEAYY4xxZQHCGGOMKwsQxnJRMIQAAAAJSURBVBhjXP1/I2djixUSt8EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#画图观察\n",
    "fig, ax = plt.subplots()\n",
    "ax.scatter(y_test, y_test_pred_ridge)\n",
    "ax.plot([y_test.min(), y_test.max()], [y_test.min(), y_test.max()], 'k--', lw=4)\n",
    "ax.set_xlabel('Measured')\n",
    "ax.set_ylabel('Predicted')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>12. 重复步骤7-11，选择验证集上MSE或RMSE最小的模型作为模型输出</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>13. 保存模型，加载模型做预测,第一种方法</h1>\n",
    "    * 使用python pickle API 保存\n",
    "    * 使用python 加载pickle文件预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#保存模型方法1\n",
    "\n",
    "# 保存成python支持的文件格式pickle, 在当前目录下可以看到regression.pickle\n",
    "import pickle\n",
    "with open('regression.pickle', 'wb') as fw:\n",
    "    pickle.dump(ridge, fw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[444.72624337]\n",
      " [442.43179818]]\n"
     ]
    }
   ],
   "source": [
    "#加载并预测\n",
    "\n",
    "# 加载regression.pickle\n",
    "with open('regression.pickle', 'rb') as fr:\n",
    "    new_ridge = pickle.load(fr)\n",
    "    test_data = np.array([\n",
    "        [(1- 19.708303)/7.441593, (1- 54.403151)/12.747985, (1- 1013.189242)/5.898733,(1- 73.220410)/14.625541],\n",
    "        [(2- 19.708303)/7.441593, (2- 54.403151)/12.747985, (2- 1013.189242)/5.898733,(2- 73.220410)/14.625541]\n",
    "    ])\n",
    "    result = new_ridge.predict(test_data)\n",
    "    print (result)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>13. 保存模型，加载模型做预测,第二种方法</h1>\n",
    "    * 使用sklearn joblib API 保存\n",
    "    * 使用sklearn joblib API 加载sklearn模型预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['regression.pkl']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#保存模型方法2\n",
    "\n",
    "#保存成sklearn自带的文件格式\n",
    "from sklearn.externals import joblib\n",
    "joblib.dump(ridge, 'regression.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[444.72624337]\n",
      " [442.43179818]]\n"
     ]
    }
   ],
   "source": [
    "new_ridge2 = joblib.load('regression.pkl')\n",
    "result2 = new_ridge2.predict(test_data)\n",
    "print (result2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>13. 保存模型，加载模型做预测,第三种方法</h1>\n",
    "    * 保存为PMML文件\n",
    "    * 使用PMML Java API加载模型预测    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PMMLPipeline(steps=[('regresser', Ridge(alpha=0.1, copy_X=True, fit_intercept=True, max_iter=None,\n",
       "   normalize=False, random_state=None, solver='auto', tol=0.001))])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#保存模型方法3,跨平台用，可以用于线上Java调用\n",
    "from sklearn2pmml.pipeline import PMMLPipeline\n",
    "from sklearn2pmml import sklearn2pmml\n",
    "regression_pipeline = PMMLPipeline([(\"regresser\", Ridge(alpha = 0.1))])\n",
    "regression_pipeline.fit(X_train_norm, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[444.72624337]\n",
      " [442.43179818]]\n"
     ]
    }
   ],
   "source": [
    "result3 = regression_pipeline.predict(test_data)\n",
    "print (result3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用PMML Java API加载模型预测的实例参见：\n",
    "\n",
    "[用PMML实现机器学习模型的跨平台上线](https://www.cnblogs.com/pinard/p/9220199.html)\n",
    "\n",
    "[PMML跨平台上线相关实例代码](https://github.com/ljpzzz/machinelearning/blob/master/model-in-product/sklearn-jpmml)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
